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Fundamentals of educational planning 



The booklets in this series are written primarily for two types of 
clientele: those engaged in educational planning and administration, 
in developing as well as developed countries; and others, less 
specialized, such as senior government officials and policy-makers 
who seek a more general understanding of educational planning 
and of how it is related to overall national development. They are 
intended to be of use either for private study or in formal training 
programmes. 

Since this series was launched in 1967 practices and concepts 
of educational planning have undergone substantial change. Many 
of the assumptions which underlay earlier attempts to rationalize 
the process of educational development have been criticized or 
abandoned. Even if rigid mandatory centralized planning has now 
clearly proven to be inappropriate, this does not mean that all forms 
of planning have been dispensed with. On the contrary, the need for 
collecting data, evaluating the efficiency of existing programmes, 
undertaking a wide range of studies, exploring the future and 
fostering broad debate on these bases to guide educational policy 
and decision-making has become even more acute than before. One 
cannot make sensible policy choices without assessing the present 
situation, specifying the goals to be reached, marshalling the means 
to attain them and monitoring what has been accomplished. Hence 
planning is also a way to organize learning: by mapping, targeting, 
acting and correcting. 

The scope of educational planning has been broadened. In 
addition to the formal system of education, it is now applied to all other 
important educational efforts in non-formal settings. Attention to the 
growth and expansion of education systems is being complemented 
and sometimes even replaced by a growing concern for the quality 
of the entire educational process and for the control of its results. 
Finally, planners and administrators have become more and more 
aware of the importance of implementation strategies and of the 
role of different regulatory mechanisms in this respect: the choice 
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of financing methods, the examination and certification procedures 
or various other regulation and incentive structures. The concern of 
planners is twofold: to reach a better understanding of the validity of 
education in its own empirically observed specific dimensions and 
to help in defining appropriate strategies for change. 

The purpose of these booklets includes monitoring the evolution 
and change in educational policies and their effect upon educational 
planning requirements; highlighting current issues of educational 
planning and analyzing them in the context of their historical and 
societal setting; and disseminating methodologies of planning 
which can be applied in the context of both the developed and the 
developing countries. 

For policy-making and planning, vicarious experience is a 
potent source of learning: the problems others face, the objectives 
they seek, the routes they try, the results they arrive at and the 
unintended results they produce are worth analysis. 

In order to help the Institute identify the real up-to-date issues 
in educational planning and policy-making in different parts of the 
world, an Editorial Board has been appointed, composed of two 
general editors and associate editors from different regions, all 
professionals of high repute in their own field. At the first meeting 
of this new Editorial Board in January 1990, its members identified 
key topics to be covered in the coming issues under the following 
headings: 

1 . Education and development. 

2. Equity considerations. 

3. Quality of education. 

4. Structure, administration and management of education. 

5. Curriculum. 

6. Cost and financing of education. 

7. Planning techniques and approaches. 

8. Information systems, monitoring and evaluation. 

Each heading is covered by one or two associate editors. 

The series has been carefully planned but no attempt has been 
made to avoid differences or even contradictions in the views 
expressed by the authors. The Institute itself does not wish to impose 
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any official doctrine. Thus, while the views are the responsibility of 
the authors and may not always he shared hy UNESCO or the IIEP, 
they warrant attention in the international forum of ideas. Indeed, 
one of the purposes of this series is to reflect a diversity of experience 
and opinions hy giving different authors from a wide range of 
backgrounds and disciplines the opportunity of expressing their 
views on changing theories and practices in educational planning. 

The time when higher education was delivered in a few 
renowned state universities is long gone. The globalized economy 
has brought strong demand for higher studies among students and 
employers, which has led to expansion, diversification and increased 
privatization of higher education. Quality control mechanisms are 
needed to regulate this new international market of higher education 
institutions. Eurthermore, the users of the system - students, 
employers and governments - want validation systems, meaningful 
qualifications, and transparent information on what the different 
institutions deliver. 

This booklet summarizes the results of one of IIEP’s projects. 
It lays out in a comprehensive manner the options that governments 
may consider when organizing a system of accreditation of higher 
education institutions. It should be of great interest for policy-makers 
and planners concerned with the organization of the higher 
education sector. IIEP is grateful to the authors Michaela Martin 
and Antony Stella for contributing their knowledge and insight 
to produce this new edition in the Eundamentals of Educational 
Planning series. 

Mark Bray 
Director, IIEP 
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Preface 



Globalization has profoundly transformed the functioning of the 
world economy. Producing new knowledge and investing in the 
production of high level manpower has become one of the main 
objectives of any country wanting to compete in the globalized 
market. At the same time the explosion of new information and 
communication technologies has made possible the emergence of 
knowledge societies, where higher education institutions play a key 
role. Development of the higher education sector has thus become a 
priority in the most advanced as well as in emergent economies. 

Higher education systems are expanding fast in the majority of 
countries. At the same time as new universities have been created, the 
higher education sector has been diversified: Different establishments 
such as community colleges, teacher training colleges, polytechnics, 
open universities and distance education centres have mushroomed 
outside traditional universities. Constraints on government spending 
have also pushed governments to consider other modes of financing 
than the public sector: Hence the number of private universities and 
management schools has exploded, indicating a very large demand 
for higher studies. 

In this globalized world, more and more students travel abroad 
to study. Foreign universities invest in developing countries and use 
their good reputation to create branch campuses abroad or to franchise 
their higher education programmes, with the hope of attracting the 
best students and to generate income. Hence marketization of higher 
education has become a reality. Families and students tend to become 
customers trying to optimize their investment without always having 
the necessary information on which to base their decision. Employers 
find themselves confronted with a plethora of new credentials whose 
value they cannot assess. In such a diversified context, the role of 
the state is to regulate the market, to create transparency, to ensure 
quality and to inform the different stakeholders. One way to do so is 
to set up quality assurance mechanisms in the education system. 
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However, setting up such a system is not easy. All over the 
world there is a great deal of variation in the approaches and models 
adopted. The situation is quite different in very large countries with 
large numbers of institutions to that in small ones. But how does one 
define the quality of higher education to start with? And who should 
do this? The ministry or a specialized public agency? Should it assess 
individual institutions or programmes? How is it possible to impose 
such a system on institutions who are traditionally very autonomous 
and resist any interferences from the government? These and many 
other questions are raised in this booklet. 

Many of these models are too new to be able to conclude in a 
firm way what is functioning best in different countries. It is however 
important in such a new field to inform the educational policy-makers 
and planners on what exists and what the options are. This booklet 
has been prepared by Michaela Martin from the International 
Institute for Educational Planning (IIEP) and Stella Antony from 
the Australian Quality Assurance Agency. It summarizes the results 
of a research project carried out at IIEP on methodological and 
organizational options in quality assurance systems. It lays out the 
options in a comprehensive manner, highlighting their characteristics 
and mechanisms. IIEP is grateful to the authors for their valuable 
contribution on this complex subject. 

Pran9oise Caillods 
Co-General Editor 



12 



International Institute for Educational Planning http://www.unesco.org/iiep 



Contents 



Preface 1 1 

List of abbreviations 15 

List of tables 17 

List of boxes 17 

Introduction 19 

I. Placing external quality assurance (EQA) 

in overall quality assurance systems 23 

General trends in bigber education systems 

that heighten the need for EQA 23 

Alternative policy instruments for regulating 

academic quality 27 

Place of EQA in the overall quality assurance system 29 

Defining quality: a complex and multi-dimensional 

concept 30 

Definitions: quality assurance, quality control, 

evaluation/assessment, audit and accreditation 33 

II. Major organizational and methodological choices 

in EQA systems 41 

Overall purposes of EQA 41 

Compulsory versus voluntary quality assurance 43 

Eitness for purpose versus standards-based approach 45 

Minimum standards versus high-level standards in 
accreditation 49 

Accreditation versus assessment versus quality audit 5 1 

Public institutions and/or the private higher education 
sector 54 

University and/or non-university sector institutions 54 

Institutions and/or programmatic EQA 56 

Use of quantitative versus qualitative data for 
accreditation 60 



International Institute for Educational Planning http://www.unesco.org/iiep 



Contents 



III. Common core elements and approaches 63 

The basic three-stage process 63 

Areas of quality assessment 68 

Approaches to quality assessment 70 

IV. The structure for quality assurance: the agency 79 

Administrative affiliation of the quality assurance agency 80 
Governance and organizational structure 82 

Resources of quality assurance agencies 87 

Credibility and accountability of the agency 9 1 

Conclusion 101 

References 107 

Web resources 110 



14 



International Institute for Educational Planning http://www.unesco.org/iiep 



List of abbreviations 



AACCUP 

AICTE 

APENS 

AUQA 

CEPES 

CESU 

CHEA 

CICA 

COPA 

CNA 

CNE 

CONEAU 

COU 

ENQA 

EQA 

GATS 

HAC 

HEI 

HEQC 

lAUP 

IIEP 

INQAAHE 



Accrediting Agency of Chartered Colleges and 
Universities in the Philippines 
All India Council of Technical Education 
Association of Professional Engineers of Nova 
Scotia (Canada) 

Australian Universities Quality Agency 
European Centre for Higher Education (located in 
Bucharest, Romania) 

National Council for Higher Education (Columbia) 
Council for Higher Education Accreditation (United 
States) 

Canadian Institute of Chartered Accountants 
Council of Postsecondary Accreditation 
National Council for Accreditation (Colombia) 
Comite national d’ evaluation [National Evaluation 
Committee] (Erance) 

National Evaluation and Accreditation Council 
(Argentina) 

Council of Ontario Universities 
European Network for Quality Assurance 
External quality assurance 
General Agreement on Trade in Services 
Hungarian Accreditation Committee 
Higher education institution 
Higher Education Quality Committee (South 
Africa) 

International Association of University Presidents 
International Institute for Educational Planning 
International Network of Quality Assurance 
Agencies in Higher Education 



15 



International Institute for Educational Planning http://www.unesco.org/iiep 



List of abbreviations 



MERCOSUR 

MOE 

NAAC 

NAC 

NBA 

NCA 

NCAC 

NWA 

OCGS 

PAASCU 

PACU-COA 



QA 

TNAC 

UGC 

UNESCO 

WASC 

WTO 



Common Market of South America 
Ministry of Education 

National Assessment and Accreditation Council 
(India) 

National Accreditation Committee (Hungary) 
National Board of Accreditation (India) 

North Central Association (United States) 

National Council of Accreditation in Colombia 
Northwest Association (United States) 

Ontario Council on Graduate Studies 
Philippine Accrediting Association of Schools, 
Colleges and Universities 

Philippine Association of Colleges and Universities 
- Commission on Accreditation and the Association 
of Christian Schools, Colleges and Universities 
Quality assurance 

Temporary National Accreditation Committee 
(Hungary) 

University Grants Commission 

United Nations Educational, Scientific and Cultural 

Organization 

Western Association of Schools and Colleges 
(United States) 

World Trade Organization 



16 



International Institute for Educational Planning http://www.unesco.org/iiep 



List of tables 



Table 1.1 
Table 1.2 
Table 1.3 
Table 2.1 



Alternative policy instruments for assuring academic 
standards 

Matrix of quality assurance functions in a higher 
education system 

Uses of accreditation systems for different 
stakeholders 

Classification of basic options used in EQA systems 



List of boxes 

Box 1. Compulsory accreditation in Hungary 
Box 2. Standards-based approach used by the regional 
accreditation agencies in the United States 
Box 3. Standard-setting at the Comite national d’evaluation 
(CNE) in Erance 

Box 4. Accreditation of high quality in Colombia 
Box 5. Quality audit in Norway 

Box 6. Elexibility in the United States accrediting system 

Box 7. Request for compulsory accreditation of 
state-approved programmes in Argentina 

Box 8. Assessing a criteria-based quality model in India 

Box 9. Adoption of the three-stage model of EQA in India 

Box 10. Standards for quality assurance in the Philippines 

Box 1 1 . Move towards competency-based standards of 

professional bodies (United States) 

Box 12. The competency map (Canada) 

Box 13. Variables and indicators of a characteristic (Colombia) 
Box 14. Quantification to guide peer assessment: NBA (India) 



17 



International Institute for Educational Planning http://www.unesco.org/iiep 



List of tables and boxes 



Box 15. 
Box 16. 

Box 17. 

Box 18. 

Box 19. 
Box 20. 

Box 21. 

Box 22. 
Box 23. 



Indicative scope of AUQA (Australia) 

Association of Professional Engineers of Nova Scotia 
(Canada) 

Governance structure of a quality assurance agency 
(Ireland) 

Governance structure of a quality assurance agency 
(Chile) 

Responsibilities of the governing body (Australia) 

Resources needed for the quality assurance agency 
(United States) 

Basic standards for accrediting accreditation agencies 
by the German Akkreditierungsrat 
Guidelines for good practices: INQAAHE 
Standards and guidelines for quality assurance 
in the European higher education area: ENQA 



International Institute for Educational Planning http://www.unesco.org/iiep 



Introduction 



Policy-making and management of higher education have 
traditionally been concerned with the maintenance and enhancement 
of academic standards and processes. The expansion, diversification 
and privatization of higher education systems worldwide have 
brought with them an increased concern with the quality of higher 
education, in both developed and developing countries. In addition, 
globalization is widely affecting higher education and is creating 
new challenges for its regulation. An increasing number of students 
travel abroad to study in foreign countries. In addition, institutions 
and programmes move across borders in the form of e-learning, 
franchising or branch campuses, and add considerably to the 
traditional offerings of local public higher education institutions. 

While the provision of higher education is becoming ever more 
diversified, increased mobility among professionals requires greater 
standardization among qualifications so that they can be assessed 
by national authorities for decisions relating to recognition. New 
instruments to assess qualifications are also needed to combat the 
academic fraud that accompanies diversification of higher education. 
Within this context of change, new methods of quality assurance 
such as accreditation systems have become a concern in higher 
education policy agendas. 

Quality assurance mechanisms used to be highly dependent 
on national administrative traditions, but nowadays there is a 
convergence of mechanisms. Higher education systems located in 
centralized administrative systems used to emphasize bureaucratic 
input steering, while decentralized systems left much of the 
authority to higher education institutions to put in place their own 
quality assurance system. Current manifestations of external quality 
assurance (EQA) seem to address the shortcomings of the traditional 
mechanisms for quality assurance found in both administrative 
realities: While national authorities may lack the competence to 
make judgements about the quality of academic programmes and 
institutions, there is little comparability of standards when academic 
institutions are both providers and judges of their own services. For 
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this reason, external quality assurance is most commonly organized 
through the creation of independent administrative structures, 
i.e. agencies that commonly function as professional buffer 
organizations between public authorities and higher education 
institutions. 

EQA mechanisms commonly focus on the perceived 
shortcomings of traditional quality assurance mechanisms. One 
of their main concerns is the quality of teaching and/or academic 
programmes, whereas earlier instruments tended to concentrate on 
the quality of research. Another growing concern is with outputs, 
linked to an overall change in the state’s steering policy. Governments 
tend to delegate more autonomy to these institutions and request 
‘accountability’, i.e. a demonstration of valuable outcomes. Finally, 
there is also a new emphasis on periodic assessment of teaching, 
based on the premise that quality is not a static condition and that it 
needs to be assessed and enhanced regularly. 

Although EQA systems tend to share similar interests and goals, 
they can take different forms. Given that a considerable number of 
countries dispose of an EQA mechanism, much experimentation has 
already taken place. A comparative analysis of systems demonstrates 
the many options in organizing EQA. These options depend largely 
on the overall purpose of the system in the regulation of higher 
education, and its particular role in the overall national system of 
quality assurance. 

This booklet aims to provide policy advice and support to 
decision-makers and managers in the area of quality assurance in 
higher education. In order to do so, the authors describe and discuss 
major issues that decision-makers in charge of quality assurance need 
to consider for the creation or development of an EQA mechanism. 
Under each issue, different options are identified that have been 
derived from a comparative analysis of quality assurance systems 
in a multitude of countries across continents. The implementation 
of such options in a given national context is illustrated through 
examples of various countries’ experiences. 

The authors begin by presenting the international trends that 
drive countries to adopt an EQA system. They then go on to discuss 
the overall purposes of an EQA system and how they relate to basic 
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organizational and methodological approaches. The authors also 
discuss variations in the definition of higher education quality and 
how it can he measured. Finally, the issue of the support structure for 
EQA is tackled in terms of administrative affiliation, management 
and needed resources. The authors conclude hy exploring ways to 
control the quality of the EQA systems themselves. 
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in overall quality assurance systems 



General trends in higher education systems that heighten 
the need for EQA 

Higher education systems worldwide are currently heavily 
affected hy a number of common trends, frequently interrelated, 
which request ongoing change and adaptation, both at the national 
and institutional levels. 

Growing social demand and expansion of systems 

Social demand for higher education has been on the increase 
over the past decades. This has resulted in increased enrolments, 
although the financial capacity has often been inadequate. During 
the period 1990/1991 to 2001/2002, the overall number of students 
worldwide increased from 68.6 million to 110.7 million (Sanyal 
and Martin, 2006). In order to enrol increasing numbers of students, 
systems have become more diversified through the development of 
a non-university post- secondary sector, and through the growing 
provision of distance education offerings. 

Privatization of higher education 

Given a frequently insufficient financial capacity to respond to 
the social demand for higher education, many countries worldwide 
that were formerly committed to exclusive public systems of 
education have adopted legislation that allows for the development 
of private higher education. This has led to tremendous growth in 
the private provision of higher education. In addition, public higher 
education institutions have also undergone a major privatization 
process through a growing reliance on cost-sharing arrangements 
and income-generation measures. 

Privatization of higher education is supported by a growing 
common understanding that the benefits of higher education largely 
accrue to the individual, especially at the professional degree 
level. In particular, higher education for working adults (lifelong 
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learning) has been targeted by both public and private higher 
education institutions as both a national and an international market. 
Consequently, specific segments of higher education (professional 
higher education for adults) are considered in many countries as 
market goods, and certain governments actively encourage their 
public higher education institutions to become active players in this 
domain. 

The ‘new public management’ concept 

In many countries, governments have redefined their roles as 
public authorities under the ‘new public management’ concept, 
which suggests - as one of its pillar ideas - greater reliance on the 
deregulation and decentralization of power from the government 
or its agencies to institutions. In many countries, deregulation has 
become part of a broader reform of public organizations, where 
the decentralization of decision-making, incentives for units and 
individual staff, negotiation of objectives and targets, as well as 
output control and a funding system based on output measures are 
the predominant tools. 

In the area of higher education, the new public management 
concept suggests the so-called ‘indirect steering’ of higher education, 
that authorities increasingly limit their role to setting guidelines and 
providing resources and incentives, and that institutions and the state 
negotiate more detailed objectives in terms of activities and outputs. 
However, the demand for accountability and trust is also raised on 
political agendas, together with greater freedom for higher education 
institutions to take decisions at their own discretion. Governments 
expect institutions to be accountable to their students and the 
public at large for the quality of their services and the utilization of 
resources. Consequently, institutions in many countries worldwide 
have been urged to provide information on their results as well as on 
their expenditure. 

The new public management concept also suggests greater 
reliance on market mechanisms; a strategy that some countries have 
been implementing for a long time, and that others have adopted 
more recently. In highly diversified and market-oriented systems 
(such as in the United States), the provision of information to the 
public through accreditation is a long-standing practice. Consumers 
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and the public at large, in addition to governments, are requesting 
market transparency, hence public information on institutional 
performance. Requests by students and parents for information 
regarding institutional performance have also become apparent in 
other regions that are becoming steered by market forces more than 
ever before. 

Globalization and international trade agreements 

Higher education systems are also increasingly affected by 
globalization, but institutions are themselves becoming actors of 
globalization. Growing potential for the international movement 
of goods, capital and persons - facilitated by recent advances in 
regional integration processes and trade agreements, as well as 
information and communication technology - have in many respects 
widely affected the structure, content and delivery systems of higher 
education worldwide. Regional trade agreements have certainly led 
to an unprecedented level of mobility in some professions, and in 
others, their globalization has significantly enhanced professional 
mobility. This has put greater pressure on governments to compare 
their national educational standards with those of other countries, and 
on institutions to confer recognized qualifications on an international 
labour market. 

In addition to professionals, students have also become more 
mobile. The demand for higher education in many countries is 
generating unprecedented growth in the number of foreign students. 
The Global Student Mobility Report 2025 (Bohm, Davies, Meares 
and Pearce, 2002), prepared by IDP Education Australia, predicted 
that the number of students seeking an education in a foreign country 
would increase from 1.8 million in 2000 to 7.2 million in 2005, the 
majority of which would be Asian students. 

International student mobility has also intensified because 
of regional integration processes which have an ever-increasing 
impact on education systems. The Bologna Process in the European 
region, for instance, aims at establishing by 2010 a common 
qualification structure in the so-called European Higher Education 
Area (Bachelor’s, Master’s and PhD), a credit transfer system, and a 
national accreditation mechanism, all of which would jointly aim at 
facilitating the mobility of students and professionals. It is expected 
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that the establishment of national accreditation systems will facilitate 
the mutual recognition of credentials. An experimental common 
accreditation mechanism has been created within the framework 
of Mercosur in South America, where professional experts from 
member states (Argentina, Brazil, Chile, Paraguay and Uruguay) 
and associate states (Bolivia, Chile, Colombia, Ecuador, Peru and 
Venezuela) were brought together to establish common standards in 
medicine, engineering and agronomy, to be applied throughout the 
region. 

GATS and borderless markets for higher education 

A direct manifestation of higher education globalization is 
the continuous expansion of ‘transnational higher education’, 
as institutions, study programmes, courses and other provisions 
of education (including distance education and e-leaming) offer 
their services to learners from different countries. Transnational 
education is increasingly conducted with a commercial aim that is 
rapidly changing. Indeed, the topic of educational services is being 
discussed in the framework of the General Agreement on Trade in 
Services (GATS) of the World Trade Organization (WTO), and has 
raised awareness about the fact that higher education has become a 
global market commodity for which there is currently no regulatory 
regime at the international level. 

International market for quality assurance services 

Globalization of education services also calls for globalization 
of quality assurance and accreditation services. Indeed, many of 
the private accreditation agencies operating at the national level, 
particularly those in professional areas of study such as management 
or engineering, offer their services to organizations located in 
countries other than their own. There have been attempts to build 
new organizations (for instance the Global Alliance for Transnational 
Education) specifically for this purpose, which offer their services at 
an international level. Some of the United States-based regional and 
professional accreditation agencies have accredited programmes 
in as many as 40 countries. The United States Council for Higher 
Education Accreditation (CHEA) has released an updated database 
of all institutions/programmes accredited by its members. This 
database provides examples of United States accreditation obtained 
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by higher education institutions in 3 1 countries/territories outside the 
United States, including Australia, Canada, Germany, New Zealand, 
Singapore and the United Kingdom (see www.obhe.ac.uk/cgi-bin/ 
news/article.pl?id=377). 

Alternative policy instruments for regulating academic 
quality 

The above trends demonstrate why concern with the quality 
of higher education, and subsequently how to assure it, has gained 
considerable importance on the policy agenda of national authorities 
of higher education. However, it must be acknowledged that quality 
assurance of higher education is by no means a new practice or 
request. Traditionally, all systems of higher education have an 
established mechanism for controlling academic activities. However, 
the nature and extent of these mechanisms vary widely according 
to different higher education systems. Those strongly influenced by 
the state (e.g. in continental European countries as opposed to the 
United Kingdom) have traditionally operated through an ex ante 
control of inputs - such as yearly item-line budgets, civil- servant 
status of academic staff with control over entry qualifications, and 
state -regulated admission systems. Educational processes used to be 
controlled through approval procedures of new curricula, detailed 
prescriptions of workload and minimum content, as well as types of 
examinations to be conducted. 

At the institutional level, evaluations of the academic 
performance of individual scholars - in particular their research 
performance - is conducted mainly by heads of departments or 
departmental committees, or by disciplinary committees at the 
national level. Performance is measured generally by the number and 
quality of publications. Such staff appraisal is conducted in relation 
to decisions on either promotion or recruitment. The assessment of 
research outcomes within the community of scholars, usually done 
informally through peer assessment, is a rather well established 
practice in most higher education systems. 

The newly emerging concern and pressure for EQA, notably 
from governments or international funding agencies, have brought 
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with them three innovative procedural changes in the assessment 
process of academic quality. 

• First, it refers to areas of academic life that previously involved 
little intervention from governments or funding agencies. In 
particular, a special interest in the quality of teaching/learning 
has strongly emerged in countries where governments either 
create mechanisms to investigate teaching/learning conditions 
or encourage institutions to set up their own mechanisms for 
assuring that the established standards are met. 

• Second, national authorities or institutions request that 
assessments he conducted on a regular basis (no longer on an 
ad hoc basis for certain types of decisions). 

• Third, the current movement in quality assurance is that it is 
becoming more concerned with outcomes than with inputs and 
throughputs. In some Latin American countries (such as Brazil, 
Colombia and Mexico), public tests have been created to assess 
the knowledge and competencies of university graduates in 
selected study areas. This measure has been introduced to create 
more transparency as to the real performance of the numerous 
public and private higher education institutions. 

Governments have a broad range of policy approaches to 
influence academic standards. Based on a theoretical notion phrased 
by Clark (1983), the three co-ordinating powers in higher education 
- the academic oligarchy, the state and the market - broadly speaking 
cover the following three important functions: 

• Direct monitoring of the quality of institutions and programmes 
by the state. 

• Provision of incentives to professional organizations for their 
self-regulation. 

• Reliance on market mechanisms for the improvements of 
academic quality. 

Table 1.1 suggests a matrix, developed by David D. Dill, of 
alternative instruments for the regulation of academic quality. 
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Table 1.1 Alternative policy instruments for assuring 
academic standards 



Focus 




Locus of influence 






Professional 

self-regulation 


State (direct) 
regulation 


Market regulation 


Research 


Professional peer 
judgement 


Research assessment 


Competitive allocation 
of research funds by 
the state 


Teaching/ 

learning 


Professional 

disciplines/ 

organizations 

Professional 

certification/licensure 

External examining 

systems 

Voluntary 

accreditation 


Assessment 

‘regulation’ 

Academic audits 
Subject assessment 
State certification/ 
licensure 
State-conducted 
accreditation 
Performance 
indicators 

National examinations 


Student-based funding 
and tuition fees 
Information provision 



Source: Dill, 2003: 5. 



As can be seen from Table 1.1, the various approaches to regulating 
academic quality depend on the players in the higher education 
system: the state, the academic oligarchy and professionals, and the 
market. The approaches are usually determined by a governmental 
authority so that they are in harmony with a prevailing philosophy 
for managing higher education. The change from one approach to 
another is also indicative of systems that wish to strengthen one of 
the three players, usually to the detriment of another. 

Place of EQA in the overall quality assurance system 

Quality assurance systems consist of a continuum of mechanisms 
that build on each other. Consequently, they evolve as a system, 
and a change in any one of them naturally has repercussions on the 
others. When new processes for the assessment of quality in higher 
education are developed, they usually form part of the processes that 
are already in place in a system, some of which may be proven to be 
weak or problematic. 
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Quality assurance systems relate to institutions and programmes 
(both undergraduate and graduate). They also address different 
functions, such as: (a) assessments related to the initiation of 
programmes and institutions (in this case commonly called 
‘licensing’, which, when successful, declares the programme or 
institution a publicly recognized entity); (b) supervision of their 
current functioning (usually with regard to minimum standards, 
including supervision of administration and finance) ; (c) accreditation 
(frequently for high levels of quality); (d) professional certification 
of graduates; and finally (e) the provision of information regarding 
recognition and accreditation of institutions and programmes. These 
functions are not necessarily present in all systems, but if they are 
not, the quality assurance will not be as effective as it could be. 

Different entities (governmental or non-governmental) may be 
in charge of carrying out these different functions. The main actors 
are typically ministries of education (whose prime responsibility 
is quality assurance); bodies with delegated authority, such as an 
inspectorate (less frequent in higher education than in other levels 
of education); more recently para-statal or private entities, such as 
quality assurance agencies; and finally professional bodies, when a 
system requires professional certification. 

Commonly, the quality assurance continuum can be presented 
in a matrix, as in Table 1.2, with the following distribution of 
responsibilities among the different actors. 

Defining quality: a complex and multi- dimensional 
concept 

The concept of quality is much disputed in higher education 
and often used by stakeholders in order to legitimize their specific 
vision or interests. Two reasons can be given for the difficulties 
encountered when defining quality in higher education: 

• There is no consensus on the exact obj ecti ves of higher education 

- although some objectives can be named, such as to produce a 
qualified work force, to train people for a career in research, to 
provide efficient management of the teaching profession, and to 
enhance life prospects. 
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• Higher education, like any type of education, is a 
multi-dimensional and complex process, which is based on the 
relationships between and among teachers and learners. It is 
difficult to grasp the interaction of inputs and throughputs and 
the exact determinants of outputs. 



Table 1.2 Matrk of quality assurance functions in a higher 
education system 



Unit of 

assessment/ 

function 


Institutions 


Programmes Programmes 
(undergraduate) (graduate) 


Students 


Opening 


MOE 


MOE or 


MOE or 


Admission test 


Initial 


Specialized 


licensed HEI 


licensed HEI 




assessment 

(licensing) 


public agency 








Supervision 


Inspectorate 


HEI 


HEI 


HEI 


Accreditation 


Buffer 


Buffer or 


Buffer or 


Outcome 




organization or 


agency 


agency 


assessment 




QA agency 


Professional 


Professional 


among 

graduates 






organization 


organization 




Professional 




Professional 


Professional 


Professional 


certification 




organizations 


organizations, 
peer review 


organizations 


Public 


MOE and/or 


MOE and/or 


MOE and/or 




information 


agency 


agency 


agency 





Source: Lemaitre, in World Bank, 2003: 106. 



As higher education becomes more inclusive and the student 
population becomes more heterogeneous, the demand on higher 
education institutions (HEIs), and for the provision of courses, 
grows increasingly more diverse. Thus, what might seem to be an 
adequate definition of quality for one type of course or institution 
may be quite inadequate for others. 

Standards versus fitness for purpose 

A basic distinction can be made between two conceptions of 
higher education quality. The first establishes that it is possible to 
identify and to quantify certain aspects of higher education, and that 
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the same standards or criteria can be applied to all courses or all 
institutions. This conception is generally based on the view that higher 
education institutions have very specific objectives and that there 
are golden standards to be maintained. The second conception of 
quality is based on the assumption that institutions of higher learning 
have different missions and objectives. Even though they may aim 
predominantly for high-level academic research, some are oriented 
towards high-level teaching, and others offer higher education to 
ethnic minorities, etc. Such a concept of quality relates to fitness for 
purpose. Consequently, the measurement of quality does not allow 
for common, quantifiable criteria, but must then rely heavily on 
specific contextual analysis. Quality, according to this conception, 
will mean, above all, that set objectives have been achieved. This 
conception implies that higher education institutions all have their 
own traditions; they are located in different environments and may 
be responsive to the needs of different communities. 

Nevertheless, the fitness for purpose approach requires some 
measure of fitness o/ purpose - that is, the recognition that not every 
purpose is acceptable, but that the purpose has to be set within what 
is understood as being at the higher education level. 

More recently, and as part of a growing concern with the 
international competitiveness of higher education systems, there 
seems to be a shift from the earlier predominance of the fitness for 
purpose approach towards a more balanced view which favours a 
standards-based approach as a basic requirement, but leaves ample 
room for institutions to fulfil those standards in the way that is most 
appropriate to their own objectives and priorities. 

The standards-based approach may imply two objectives: It 
may aim at ensuring that (a) minimum quality standards are met 
in all institutions; and (b) programmes and institutions performing 
below those predefined standards are closed down. This approach 
may also indicate ideal standards towards which institutions should 
strive. With this latter objective, the standards-based approach 
becomes predominantly a vehicle for quality improvement. While 
most systems of quality piloting try to reflect the particular mission 
of a higher education institution, they do offer a corps of standards 
that form the basis of a quality model. 
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Who defines quality and with what interest? 

When discussing quality, it is not merely a question of setting 
standards; it is also very much an issue of who defines it on the 
basis of what interest - academics themselves, students, government 
and its agencies, or professional bodies or employers? Academics 
will judge the quality of a higher education course or institution 
according to the quality of research work done by a colleague or an 
institution. Undergraduate students will focus on the quality of the 
teaching, their learning experience and environment. Professional 
bodies tend to focus on professional standards and skills related to the 
professions that students are trained for, etc. This means, therefore, 
that any methodology for EQA has to balance the different interests 
of stakeholders in order to reach a consensus on the meaning of 
educational quality. 

One of the main tasks of a quality assurance agency is precisely 
to determine its understanding of what quality is and how to define 
it, the stakeholders it will consult, the way in which it will take into 
account international standards and definitions, and how to legitimize 
and make this definition acceptable throughout the system. 

Definitions: quality assurance, quality control, 
evaluation/assessment, audit and accreditation 

The understanding of the term ‘quality’, its conceptualization 
and operationalization, have obvious implications in any attempt to 
assure it, but it is also important to understand the other terms used 
in EQA. These terms are frequently used very loosely; therefore, the 
following pages will be dedicated to explaining some of the most 
common. There is no general consensus on the exact meaning of 
each of the terms - some are generic for the whole field, such as 
quality assurance (internal and external) and quality assessment, 
whereas others relate to more specific approaches (quality audit 
and accreditation). They relate also to the responsibility of different 
actors in the system and to different foci of attention.' Quality 



1 The International Network of Quality Assurance Agencies in Higher Education 
(INQAAHE) has been working on a glossary, which can be consulted at www. 
qualityresearchintemational.com/glossary/ 
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assurance (QA) is a generic term used as shorthand for all forms of 
external quality monitoring, evaluation or review and may he defined 
as a process of establishing stakeholder confidence that provision 
(inputs, processes and outcomes) fulfils expectations or measures up 
to the minimum requirements. 

At the institutional level, quality assurance in general is defined 
as that aspect of the overall management function that determines 
and implements the quality policy (intentions and directions of 
the organization). Procedures might be imposed on institutions 
by the government or funding agencies (e.g. for accountability 
and compliance purposes), or the institutional or departmental 
management might set them up internally. They might be part of a 
traditional process (i.e. of institutional accreditation or programme 
validation/review) or relate to new practices such as the use of 
students’ ratings of the teaching staff. They may be geared towards 
research activities, courses, academic staff, and support functions 
(e.g. administrative audit). These procedures may also analyze, in an 
aggregated manner, these functions either in individual departments 
or in an entire institution. 

Internal quality assurance refers to the policies and mechanisms 
implemented in an institution or programme to ensure that it is 
fulfilling its own purposes and meeting the standards that apply 
to higher education in general or to the profession or discipline in 
particular. 

External quality assurance refers to the actions of an external 
body, which may be a quality assurance agency or any body 
other than the institution that assesses its operation or that of its 
programmes, in order to determine whether it is meeting the agreed 
or predetermined standards. 

According to Vlasceanu, Griinberg, and PMea (2004) quality 
assurance has the following meaning: 

“Quality assurance: An all-embracing term referring to 
an ongoing, continuous process of evaluating (assessing, 
monitoring, guaranteeing, maintaining, and improving) 
the quality of a higher education system, institutions, or 
programmes. As a regulatory mechanism, quality assurance 
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focuses on both accountability and improvement, providing 
information and judgements (not ranking) through an agreed 
and consistent process and well-established criteria. Many 
systems make a distinction between internal quality assurance 
(i.e. intra-institutional practices in view of monitoring and 
improving the quality of higher education) and external quality 
assurance (i.e. inter- or supra-institutional schemes of assuring 
the quality of higher education institutions and programmes). 
Quality assurance activities depend on the existence of the 
necessary institutional mechanisms, preferably sustained 
by a solid quality culture. The scope of quality assurance is 
determined by the shape and the size of the higher education 
system. Quality assurance varies from accreditation in the sense 
that the former is only a prerequisite for the latter ... Quality 
assurance is often considered as a part of the quality management 
of higher education, while sometimes the two terms are used 
synonymously” (Vlasceanu et al, 2004: 48-49). 

A related, equally broad concept is quality assessment, which 
in many instances is a synonym of ‘evaluation’ or ‘review’. Many 
governments have started putting in place quality assessment 
mechanisms, which, rather than producing any type of grading or 
ranking of institutions, apply a set of recommendations to improve 
the quality of a given institution or programme. 

“Quality assessment/quality review [also called evaluation]: 
Indicates the actual process of external evaluation (reviewing, 
measuring, judging) of the quality of higher education 
institutions and programmes. It consists of those techniques, 
mechanisms, and activities that are carried out by an external 
body in order to evaluate the quality of the higher education 
processes, practices, programmes, and services. Some aspects 
are important when defining and operating with the concept 
of quality assessment: 1) the context (national, institutional); 
2) the methods (self-assessment, assessment by peer review, site 
visits); 3) the levels (system, institution, department, individual); 

4) the mechanisms (rewards, policies, structures, cultures); 

5) certain quality values attached to quality assessment such as 
academic values (focusing upon the subject filed), managerial 
values (focusing on staff and their teaching skills and classroom 
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practice), employment values (emphasizing graduate output 
characteristics and e-learning outcomes)” (Vlasceanu, et ai, 
2004: 48). 

A rather particular approach to quality assurance is quality 
audit, which does not assess quality or performance per se, hut the 
quality of quality assurance mechanisms. It is a method to evaluate 
the strengths and weaknesses of the quality assurance mechanism 
adopted hy an institution in order to monitor and improve the activities 
and services of a subject, programme, or whole institution. 

“Quality audit: The process of quality assessment hy which 
an external body ensures that 1) the institution or programme 
quality assurance procedures or 2) that the overall (internal 
and external) quality assurance procedures of the system are 
adequate and are actually being carried out. Quality audit looks 
to the system for achieving [or assuring] good quality and not at 
the quality itself. A quality audit can be realized only by persons 
(i.e. quality auditors) who are not directly involved in the areas 
being audited. Quality audits can be undertaken to meet internal 
goals (internal audit) or external goals (external audit). The result 
of the audit must be documented (audit report)” (Vlasceanu, 
et ai, 2004: 50). 

Accreditation is the most widely used method of EQA to be 
introduced recently in many higher education systems, either as a 
transformation of previously used methods of EQA or as an entirely 
new method. It is based on assessment and evaluation methods, 
but it makes an explicit judgement on whether a programme or an 
institution meets particular quality standards, which may be either a 
set of minimum standards, standards of high quality or excellence, 
or based on the institution’s own purposes. Accreditation, therefore, 
inevitably involves some kind of benchmarking (of what is acceptable 
and what is not) and a set of existing quality criteria. Accreditation 
is thus the only method within the quality assurance spectrum which 
makes an explicit judgement about the degree to which an institution 
or programme actually meets the pre-determined standards or 
requirements. 

Accreditation against minimum (also called ‘threshold’) 
standards assures that the quality of programmes or institutions is 
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acceptable. When it is also linked to authorization to operate, it 
is usually called licensing. Some systems also apply high quality 
standards, and this makes it possible to differentiate between those 
programmes or institutions that meet the minimum standards (and 
are thus acceptable) and those that are of excellent standard. There 
are also some systems that accredit institutions or programmes solely 
against their own standards, but these are becoming rare, unless they 
can also show that their standards meet the basic minimum standards 
for the profession or for higher education in general. 

“Accreditation is the process by which a (non-)governmental 
or private body evaluates the quality of a higher education 
institution as a whole or a specific educational programme 
in order to formally recognize it as having met certain 
predetermined minimal criteria or standards. The result of this 
process is usually the awarding of a status (a yes/no decision), 
of recognition, and sometimes of a license to operate with a 
time-limited validity. The process can imply initial and periodic 
self-study and evaluation [italics mine] by external peers. The 
accreditation process generally involves three steps with specific 
activities: 1) a self-evaluation process conducted by the faculty, 
the administrators, and the staff of the institution or academic 
programme, resulting in a report that takes as its reference the 
set of standards and criteria of the accrediting body; 2) a study 
visit conducted by a team of peers, selected by the accrediting 
organization, which reviews the evidence, visits the premises, 
and interviews the academic and administrative staff, resulting 
in an assessment report, including a recommendation to the 
commission of the accrediting body; 3) examination by the 
commission of the evidence and recommendation on the basis 
of the given set of criteria concerning quality and resulting in a 
final judgement and the communication of the formal decision 
to the institution and other constituencies, if appropriate” 
(Vlasceanu, et al, 2004: 19.) 

According to the glossary compiled by the International Network 
of Quality Assurance Agencies in Higher Education (INQAAHE) (see 
www.qualityresearchinternational.com/glossary/) it is very difficult to 
define accreditation, because the concept is very volatile and practices 
are constantly changing. While the terminology was developed in 
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the United States, it has since also been adopted in Latin America 
and Europe, with however an altered meaning and role. According 
to Chemay, accreditation assures the educational community, the 
general public and other agencies or organizations that an institution 
or programme (a) has clearly-defined and educationally appropriate 
objectives; (b) maintains conditions under which the achievement of 
these can reasonably be expected; (c) is in fact accomplishing them 
substantially; and (d) can be expected to do so in the future. 

Instead of a definition, some characteristics of accreditation are 
given: 

• Accreditation is a. formal decision. 

• Accreditation is based on an overall assessment of the HEI or 
its core activities. 

• Accreditation is based on the assessment of at least minimum 
requirements (threshold quality). 

• Accreditation concerns a yes/no/conditional decision. 

• Accreditation will have consequences, for example in the 
professional field, concerning recognition, funding and student 
aid. 

Accreditation might be seen as providing a formal quality 
certificate to an HEI or a programme showing that the HEI or 
the programme meets at least expected minimum requirements 
(Chernay, 1990). 

The information provided by accreditation systems on the 
quality of both programmes and institutions may be used by a variety 
of stakeholders in society, among which the following are the most 
important: (a) government; (b) students; (c) employers; (d) funding 
organizations; (e) institutions of higher education. 

Marjorie Peace Eenn (2004) distinguishes the ways in which 
EQA systems are used, all of which apply to accreditation systems 
in particular (see Table 1.3). 

Table 1.3 demonstrates clearly that while accreditation systems 
are put in place to assure governmental authorities and other societal 
stakeholders that higher education is performing - and will continue 
to perform adequately - they also accomplish broader purposes, 
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including those related to the management and planning of higher 
education institutions. 



Table 1.3 


Uses of accreditation systems for different 
stakeholders 


Users 


Uses 



Government To define national higher education 





To assure quality higher education 
To assure a quality labour force 

To determine which institutions and programmes receive 
public funding 

To accept into civil service only those who have 
graduated from accredited institutions 
To generally use quality assurance as a means of 
consumer protection 


Students 


To assist in selecting an institution for study 
To ensure transfer between accredited institutions 
To ensure admission at the graduate level at a different 
institution from that of the undergraduate degree 
To assist in finding employment 


Employers 


To assure qualified employees 



Funding organizations To determine eligible institutions for funding 



Higher education To improve institutional information and data 
institutions To enhance institutional planning 

To determine membership in certain organizations 
To facilitate transfer schemes 
To assure a qualified student body 



Source’. Peace Lenn, 2004. 
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The above definitions of the concepts related to EQA may give 
the impression that the methodology and organizational options 
are relatively homogeneous in practice. In spite of this apparent 
similarity, quality assurance agencies use the term ‘external quality 
assurance’ to denote different practices to serve various purposes and 
they exercise the responsibility of carrying out quality assurance in 
various ways. However, these different approaches are not clear-cut 
and, in practice, quality assurance agencies tend to combine 
different approaches in order to adapt their quality assurance 
system to the particular requirements of the local context. Many 
important differences become evident in a comparative analysis of 
methodological options. They quickly bring to the fore the fact that 
a set of basic process elements form a common structure, but that 
EQA systems vary in their underlying objectives as well as their 
basic approaches. 

Overall purposes of EQA 

EQA in higher education refers to a wide range of purposes 
and related methodological frameworks. Some of these reflect 
governmental interest and demand, whereas others more directly 
address the internal needs of institutions. Consequently, the purposes 
of EQA are closely related to the use that will be made of information 
outcomes. Accreditation in institutions of higher education usually 
serves the following purposes: quality control; accountability /public 
assurance; and improvement in teaching/learning, each of which 
will be elaborated upon in the following paragraphs. 

Quality control refers to the traditional function whereby 
governments make sure that higher education provision is in line 
with the minimum requirements for quality. When the majority of 
higher education systems were public, this function used to be less 
prominent because it was assumed that sufficient input steering would 
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produce acceptable levels of quality. This is now questioned and, 
in addition, the ongoing process of privatization (and in particular 
the proliferation of private national and international providers) has 
enhanced the need for national governments to check on minimum 
levels of quality, even if only to protect national consumers and 
make sure that the higher education provision relates to national 
development objectives in one way or another. 

Secondly, EQA geared towards accountability is commonly also 
commissioned by public authorities as part of their higher education 
policy agenda. It is frequently linked to value-for-money concerns, 
transparency and public assurance. Accountability and compliance 
concerns reflect the need to provide public information and make 
judgements about the fitness for purpose (including the fitness of 
purpose) concern, soundness, or level of satisfaction achieved. 
EQA conducted with a predominant accountability objective is 
commonly used to provide assurance to external stakeholders on 
levels of quality, acceptable or high-level standards, as well as the 
international comparability of both public and private providers 
(Harvey, 1999: 24). 

Through the quality model it uses and the setting of criteria and 
standards to be measured, accreditation is one of the standards-based 
approaches among the EQA models. It can be used by the 
government to make higher education institutions more coherent 
with policy preferences in general and recent reform initiatives in 
particular. It is certain that accreditation standards communicate a 
detailed framework of preferences against which institutions know 
they will be judged. Together with legal frameworks and funding 
methodologies, EQA has become a strong instrument for steering 
academia. This is particularly the case with regard to systems of 
reporting between institutions and governmental authorities, which 
can be highly enriched through regular provision of data and reports 
from quality assurance agencies. 

Thirdly, EQA may also be explicitly geared towards the 
improvement of existing practices. In order to achieve this purpose, it 
wifi have to rely largely on an involvement of the people responsible 
for teaching and research activities, the academic staff, be it 
individually or collectively. It seems logical to expect that EQA wifi 
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lead to improvement, partially through the compliance ohjective, 
and partially through the setting of so-called ‘high-level standards’, 
which institutions and their departments should strive to reach. 
However, the main reasons for the improvement brought about by 
EQA are the formal and systematic self-assessment procedures it 
helps establish within HEIs. It is argued that ‘transformative’ quality 
improvement occurs more easily when academics openly assess 
their true teaching. Otherwise, an EQA system may simply produce 
what has now become known as a ‘compliance culture’. 

There has been lengthy discussion over whether control, 
accountability and improvement objectives are compatible or 
mutually exclusive. While it is certainly true that EQA systems 
address them all in one way or another, they are usually geared more 
towards one than the other. 

Compulsory versus voluntary quality assurance 

One of the most important things to consider in an EQA system 
that is particularly related to its overall purpose is whether it is of 
a compulsory or a voluntary nature. A compulsory system requires 
that all institutions or programmes that fall under EQA mechanisms 
periodically undergo accreditation at specified intervals, but is 
not generally exclusively concerned with checking on minimum 
standards. Compulsory systems are thus frequently set up because 
there is a deficient or non-existent licensing scheme, or they apply to 
types of programmes in which the state has a special responsibility. 
Such programmes might include teacher training or courses that 
prepare students for professions that are vital for national development 
and security - such as medical professions, architecture and civil 
engineering. 

The more current EQA systems, however, are of a voluntary 
nature. This means that institutions and departments are under no 
obligation to undergo EQA. Their motivation to apply for EQA could 
be to obtain special status (be accredited), which would give them an 
advantage in an environment where there is competition for students, 
funding or access to specific funding (for instance student funding 
or special incentive schemes). When EQA systems are voluntary, 
it is often expected that the advantages related to EQA will create 
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an impetus, causing all, or at least a majority, of institutions to join 
the system. Voluntary EQA systems are more commonly related to 
a policy agenda of quality improvement, because institutions and 
departments can decide themselves whether or not they wish to join 
the process, depending on whether or not they wish to adhere to the 
proposed quality model. Voluntary EQA systems are also more easily 
acceptable to the higher education community because they generate 
less opposition from a usually well-organized societal force. 



Box 1. Compulsory accreditation in Hungary 

Hungary developed its procedure and organization of accreditation 
during the 1990s and made it official through government hills 
that were modified on three different occasions: the Temporary 
National Accreditation Committee (TNAC) 1993-1994, the National 
Accreditation Committee (NAC) 1994-1996 and the Hungarian 
Accreditation Committee (HAC) 1997. HAC is a ‘remote control’ 
organization and is responsible for content and quality, both of 
which are implemented by way of accreditation of programmes and 
institutions. 

HAC is responsible for different functions. It performs institutional 
accreditation of universities and colleges every eight years (with 
special regulations for church- and privately-run institutions applying 
for state recognition), whereby, in addition to the management 
and infrastructure of the institution, all degree programmes are 
assessed (through self-evaluation, the peer visit, and subsequently 
the published report). The HAC is also in charge of assessing new 
degree programmes and doctoral schools, specialized postgraduate 
programmes and 2-year post- secondary vocational programmes at 
HEIs. It licenses new universities, colleges and foreign higher education 
institutions. In addition to these functions that are directly related to 
quality management, it issues statements on national qualification 
requirements and on credit system regulations, and it evaluates drafts 
of international agreements on diploma recognition and equivalence. 
More recently, accreditation of transnational higher education and 
evaluation of academia have been added to its portfolio of functions. 

One of the particularities of the Hungarian accreditation system is 
that institutional accreditation is based on both programme assessment 
and on its already offering a minimal number of programmes that have 
been accredited with the grade of either ‘exceptional’ or ‘strong’. 
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In order to fulfil these above-mentioned functions, HAC issues 
recommendations (or decisions) at three different levels. The first 
of these levels consists of disciplinary programme committees, the 
second (medium) concerns college, institutional and interdisciplinary 
committees and the third level is the plenum of HAC. Final decisions are 
considered to be corporate actions that concern programmes, faculties 
and institutions. Thus HAC recommends that the Ministry of Education 
pronounce a decision for accreditation for undergraduate programmes 
and different faculties and institutions. It is the responsibility of HAC, 
however, to advise accreditation of postgraduate programmes. 

The objectives of the Hungarian accreditation system were initially 
geared to the principle assurance of minimum quality standards, 
in particular at the programme level where disciplinary committees 
have established minimum requirements for the accreditation of 
programmes. Consequently, accreditation is compulsory in Hungary. 
Institutional accreditation possesses, however, a component that aims 
in particular at the improvement of quality. Faculties and institutions 
must prepare a self-study report, which is followed up by a visiting 
team that examines the whole institution, as well as the self-evaluation 
studies conducted at other specific levels. There is apparent tension due 
to performing both objectives simultaneously, and critics of the system 
believe that the quality improvement function should have precedence. 
The current intention is to help institutions develop and assess their own 
quality management procedures as part of the accreditation process. 
Source: Kozma, 2003: 9-11. 



Fitness for purpose versus standards-based approach 

The above-mentioned dichotomy of objectives and fundamental 
nature (compulsory/voluntary systems) is also directly related to the 
fundamental approach used to assess quality, which is either fitness 
for purpose or standards-based. The fitness for purpose approach 
begins its analysis from the stated purpose of a higher education 
institution or a programme (mission statement), while also asking 
whether or not this purpose is an acceptable purpose of higher 
education (fitness of purpose). This approach has been heavily 
supported with the argument that institutions and programmes 
cannot all be judged against the same standards since they may 
serve specific clienteles and service groups in a diversified system 
of higher education. For instance, a traditional university located 
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in a major urban environment and that heavily emphasizes the 
excellence of its research, may not necessarily be judged against 
the same set of standards as a teaching-only institution whose aim 
is to train non-traditional student groups. However, it has also been 
argued that certain standards (in particular minimum norms) must 
be demanded from all higher education institutions, and that they 
must be held accountable for them. This is increasingly becoming 
the more common opinion, in particular within the context of a 
growing demand for international comparability and, consequently, 
more and more systems of quality assurance are moving towards a 
standards-based approach of accreditation. The fitness for purpose 
approach is usually understood as being the more appropriate 
approach for quality improvement, whereas the standards-based 
approach is more easily associated with accountability and 
compliance. 



Box 2. Standards-based approach used by the regional 
accreditation agencies in the United States 

Accrediting agencies have a long history of requiring institutions to 
provide evidence about what they do. When USA accrediting agencies 
got under way early in the twentieth century, they required very specihc 
information on the college’s structure and programmes. Standards 
were limited in number, generally relied on available information, 
and were usually quantitative. During the 1920s, for example, typical 
requirements asked for information on the number and capacity of 
classroom buildings, the number of volumes in the library, the number 
and credentials of academic staff, and the size of the annual budget. The 
purpose, generally speaking, was to ensure that academic institutions 
had adequate organizational resources or sources of stability that could 
support a quality education. Notably, the focus was on the institution, 
and not on students nor on learning and instruction. 

This approach was criticized on several grounds: first, it was 
said to give too much attention to fragmented information that, while 
‘countable’, was not necessarily meaningful. Second, it did not allow 
for differences in institutional mission and type. Third, these measures 
gave too much emphasis to ‘inputs’ or ‘resources’ rather than to what 
use was made of them. Adding to these arguments, undoubtedly, was 
the fact that, as experience accumulated with accreditation and with 
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lists of accredited institutions, it became increasingly evident that 
most institutions offered programmes of good quality and deserved 
to be accredited, even if they failed to meet certain prescriptive 
requirements. The requirements themselves were increasingly seen as 
not meaningful. 

By the 1930s, with the North Central Association taking the lead, 
the idea of a single set of standards was dropped. The NCA instead 
decided to focus on the ‘total pattern’ of an institution’s activity and to 
take into account the purposes that the institution itself had chosen. A 
small, religiously affiliated college may have very different purposes 
than one of the Mid- West’s large state universities, it was recognized, 
and should be judged according to its own purposes. 

To implement this new approach, the NCA revised its approach to 
accreditation review. Evidence gathered about an institution was to be 
assessed in terms of the overall pattern the evidence presented, instead 
of the previous emphasis on meeting each specific standard. Standards 
were from then on known as ‘criteria’ to reflect their change in purpose. 
Under this ‘holistic’ approach, an institution could be deficient in one 
area but have offsetting strengths in other areas. 

This attention to a university’s overall pattern of activity was 
adopted by the NCA and used throughout the 1930s and 1940s 
(NCA, 1997: 4). Other regional accrediting agencies took similar 
actions during this period. Notably, although they took steps to offer 
greater flexibility in how their standards were interpreted, they did 
not change their wording from ‘standards’ to ‘criteria’. Numerical 
information continued to he used hut it was given less importance. 
Qualitative judgement became more important. Still, the focus was on 
the institution, its organizational strengths and distinctive educational 
offerings and mission. 

In retrospect, it can be said that this ‘holistic’ emphasis may 
have slowed but did not stop a gradual process of increased detail in 
accrediting requirements. The wording changed, the guidance became 
more flexible, but the number of standards grew. Most accrediting 
agencies today work with quite detailed standards, intended to reflect 
both the general responsibilities that all institutions or programmes 
should meet, and the criteria by which to judge whether those standards 
are met. The Northwest Association (NWA), for example, currently 
has nine standards, with a total of 45 subparts (Northwest Association, 
1999). 

Source: El-Khawas, 2001: 60-62. 
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When the aim of EQA is to judge whether an institution or 
a programme should he accredited or not, it is necessary to use a 
standards-hased approach, as is the case in the regional accreditation 
agencies in the United States. 

For almost 20 years, the French system of quality assessment 
has been anchored in a rather open system of review, whose aim is to 
assess the particular strengths and weaknesses of higher education 
institutions, and to provide information to improve the internal 
processes at a given institution while making information available 
to the broader public. In 2003, it was decided to adopt a series of 
standards that would serve both evaluators and higher education 
institutions in their internal management (see Box 3). 



Box 3. Standard-setting at the Comite national 
d’ evaluation (CNE) in France 

At the same time, we saw that the work of the CNE had to become 
more professional to establish its reputation. We sought to strengthen 
our methodology and explain it more clearly in the international arena. 
The most successful achievement in this held came in 2003 with the 
publication of the Livre des references [Book of standards], which 
is now the framework for internal evaluation. The reasoning of this 
document is based on demonstration, leaving the university to choose 
which lessons it should draw. It addresses three areas: educational 
policy, research policy and the extent to which the university’s 
management serves its objectives. The book is divided into ten reference 
frameworks, each of which dehnes a major area of university life on 
which the expectations of users and partners are based; 63 references 
constituting an implementation system; and a non-exhaustive list of 
302 criteria, each of which formalizes a mechanism that contributes 
to achievement of the objective. The Livre des references has not only 
been used in all CNE evaluations since its publications, but has also 
proved to be an effective communication tool. 

Source: Levasseur, 2005. 



48 



International Institute for Educational Planning http://www.unesco.org/iiep 



Major organizational and methodological choices in EQA systems 



Minimum standards versus high-level standards 
in accreditation 

Accreditation, the standards-based approach of EQA, may 
apply either minimum or high-level standards. When minimum 
standards are used, which is more common, it tends to resemble a 
licensing scheme for institutions or programmes, and thus functions 
as a periodic licensing mechanism. Minimum standards usually 
address input factors relating to students, staff, buildings, facilities 
and finances, as well as process elements such as governance 
and management systems, and also research activities, which are 
perceived to form the minimum conditions under which a meaningful 
higher education institution can function. The main objective of such 
an accreditation system based on minimum standards is to enforce 
conformity with standards and accountability. 



Box 4. Accreditation of high quality in Colomhia 

Higher education in Columbia is complex and heterogeneous. It 
consists of a multi-layer system of universities, colleges, technological 
institutions as well as intermediate technical/professional institutions. 
Social demand for higher education from an increasing number of 
secondary school leavers has grown tremendously over the past two 
decades. However, due to funding constraints, public provision could 
only satisfy part of this demand, which has led to the proliferation 
of numerous private programmes and institutions that offer higher 
education with different levels of both quality and relevance. 

As part of a reform of the higher education system in Colombia 
proposed in 1992 (Law 30) the system of accreditation of Colombian 
higher education institutions was created under the auspices of the 
National Council of Accreditation (CNA), which had been specifically 
set up by this Law. CNA functions under the National Council of 
Higher Education (CESU), which is the main body for policy-making 
in Colombian higher education. CNA consists of a group of highly 
respected Colombian academics and of a secretariat that is in charge of 
co-ordinating on-going accreditation processes. 

... [CNA is in charge of] voluntary accreditation for high levels of 
quality. Accreditation of Excellence is both a voluntary and temporary 
process and its methodology stresses quality enhancement rather 
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than quality control. As a consequence, Accreditation of Excellence 
adopted a system whereby ideal characteristics of quality are compared 
in the light of reality, thus creating strong incentives for quality 
improvement. 

The methodology for the. Accreditation of Excellence of academic 
programmes and institutions is founded on a four-stage process. Once 
an institution has requested accreditation of its programmes, the first 
phase concerned with checking the eligibility of criteria may begin. 
Following this stage, the academic undergraduate programme of the 
institution undergoing accreditation conducts a self-evaluation based 
on a pre-established methodology and set of criteria and quality 
characteristics. An external peer visit of the institution then follows, 
which leads to the preparation of a report on which the institutional 
management may comment. The concluding report of the peers 
is submitted to CNA which then proceeds to the final evaluation or 
synthesis and issues a recommendation to the Ministry of Education 
to accredit the programme for a given duration ranging from three to 
ten years. 

CNA has prepared a model for the Accreditation of Excellence, 
which is based on the following seven factors (...); 

• Institutional project; 

• Students and teachers; 

• Academic processes; 

• Institutional well-being; 

• Organization, administration and management; 

• Graduate students and impact on the environment; 

• Physical and financial resources. 

The factors are described and organized into a total number of 
66 characteristics. Each characteristic includes indicators that allow 
measuring the degree of compliance with an ideal value. According to 
the object for accreditation, certain characteristics are more important 
than others, and some are considered as crucial. This leaves institutions 
and peers with a checklist of items to be interpreted with regard to 
a particular department or specific institutional circumstances. 
Institutional accreditation assessment is based on similar factors 
and characteristics but the focus is more on the institutional and 
organizational processes. 

Source: Revelo and Hernandez, 2003: 9-11. 
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Accreditation based on high standards assumes either that 
the minimum standards are already checked through a different 
mechanism or that the system shows relatively even levels of quality. 
Accreditation based on high standards is also the more predominant 
form in countries where universities enjoy a traditionally high 
level of autonomy. It is usually of a voluntary nature, because its 
main objective is to provide a model and a set of references for 
institutions to incite them to strive to achieve quality improvement. 
The shortcoming of this mechanism is that it is not very effective 
in weeding out unacceptable levels of quality, because it focuses 
on those institutions and programmes that are already of more than 
acceptable quality. It is thus not an effective mechanism to deal with 
programmes of uneven and sometimes unacceptable quality offered 
by a multitude of (commercial) providers. 

Accreditation versus assessment versus quality audit 

In the earlier discussion on basic definitions of terms used in 
quality assurance, it was pointed out that EQA systems may focus on 
the quality of activities and services themselves or on internal quality 
assurance systems. If they focus on the quality of activities and 
services, these will be judged from the point of view of institutional 
goals or based on a set of standards, which is usually the function of 
accreditation and assessment. Quality audit as defined above, on the 
other hand, is a method that judges the extent to which an institution 
has a monitoring system in place that clearly conveys its strengths 
and weaknesses. This means that an institution needs strong internal 
reporting mechanisms, as well as mechanisms for collecting data on 
teaching performance, and must collect systematic data on student, 
graduate and employer satisfaction. It also means that an institution 
has mechanisms in place to deal with low-quality programmes and 
continuously improve quality that is already good. Since it focuses on 
processes, quality audit is thus very much in line with the objective 
of improvement in EQA, but it does not lead to certihcation of or 
compliance with an expected level of quality, neither to comparability 
of such levels of quality, which is the case of standards-based quality 
assessment. If a quality assurance agency conducts quality audits, it 
assesses whether higher education institutions have a reliable and 
adequate internal quality assurance system in place. 
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Box 5. Quality audit in Norway 

In order to comply with a national concern for quality, identified in 
several national and international reports, Norway created in 2002 
a new agency for accreditation, called NOKUT. It was decided to 
adopt an audit approach through which it would be assessed how the 
institutions handle their responsibility for educational quality. 

After conducting a pilot study, the following recommendations 
were made for the development of the audit approach; 

1. Quantitative documentation at the institutional level in the form 
of key figures in the categories: access, resource input and results. 
The report listed data requirements under each category, and 
expressed the question of common reporting and data formats 
across institutions. 

2. Institutional guidelines for its quality work. The following 
elements should at least be included in these guidelines: 

• Quality plan and governance including objectives, standard 
procedures, responsibilities and administrative resources, 
leadership and governance, priority areas and action plan. 

• Registers and reports (as mentioned in point 1). 

• Evaluations done within the institution (both self-evaluations 
and external evaluations). 

• The publishing of an instimtional yearly report on quality 
work. 

3. The yearly report on educational quality. This report is to replace 
the need for a yearly self-evaluation. The report should be part of 
the internal quality system (see point 2). 

4. External audit evaluation. This is the actual audit exercise. The 
report stresses the importance of looking beyond the system, and 
into the actual functioning and effects it may have. The report 
proposes a mandate for the external audit evaluation team (NNR, 
August 2002). 

It was expected that the audit approach have the following 
strengths in the system; 

• The audit system takes into proper account the existence 
of large, autonomous and professionally run institutions of 
higher education, being able to conduct internal systems of 
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quality assurance. The institutions will themselves have the 
required insight to make systems suited for them. 

• Even if they are able to conduct such systems, there may 
be opposition within the institution in doing so. The audit 
system may in those cases put pressure on the institution in 
developing good internal systems of quality assurance. 

• Within general guidelines and standards, it gives the 
institutions freedom to develop their own internal systems 
and mechanisms. There will be few specific instructions 
and templates and no procedure manuals governing what 
the institutions must or should do in the way of monitoring 
and evaluating their programmes. Rather, the standards will 
be predominantly such as can be deduced from one simple 
question; what does it mean to have a reliable system of 
quality assurance? This open approach, it is hoped, will cater 
for flexibility, creativity, pluralism and a sense of institutional 
ownership inside a common framework. 

• The audit system can be more economical, which in turn 
may leave more resources for other types of evaluations. 

• With national audit focusing on the institution’s quality 
system and the documentation it produces, it can be more 
economical and more development-oriented. In principle, 
audits will make use of the same corpus of evaluation data as 
the institution itself does, while also ‘checking’ these data and 
the institution’s own assessment against other information 
obtained through site visit interviews and from other sources. 
Audits will be concerned with individual courses mainly 
in so far as the evaluation data show indications of failing 
quality, when they may trigger a closer inspection at subject/ 
programme level. 

• Preserving a certain space for the established tradition of 
development-oriented evaluations has been an important 
factor. An ‘open’ audit system may be looked upon as 
development-oriented in itself - in addition to providing 
accountability. Being more economical than a system of 
cyclic evaluations at programme level, it may also leave 
more resources for other type of evaluations. 

Source: Levy, 2005. 
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Public institutions and/or the private higher education 
sector 

When creating a new EQA system, public authorities have 
to make an important decision regarding coverage of the system 
with regard to both public and private institutions. This is, in many 
instances, both a legal and a political choice. In many systems, most 
public institutions are resourced by the state and may therefore be 
of more even quality than private providers. In addition, public 
institutions, in particularly universities, are often a powerful 
pressure group that may oppose the implementation of EQA in their 
own sphere. Consequently, EQA systems have been set up in some 
countries specifically to quickly develop a private sector of higher 
education institutions. In other country contexts, both governments 
and citizens expect their public institutions to show that they make 
good use of public resources, and therefore are the primary targets 
for EQA. Einally, in some other countries, the underlying philosophy 
is that both public and private higher education institutions are 
expected to contribute to the fulfilment of national policy objectives 
(i.e. human resource development, social cohesion and scientific 
and cultural development), and that a quality assurance system must 
be applied in both subsectors. 

In addition, with a forward-looking perspective, countries 
may wish to receive foreign providers of higher education in their 
country, and given the current negotiation within GATS, a decision 
might be made in the future to give private providers equal (i.e. non- 
preferential) treatment to public institutions. This would then mean 
that the basic requirements for the functioning of both public and 
private providers, with the exception of providing public funding, 
would have to be similar. This would also apply to EQA procedures 
and standards. Eor these reasons, there is a general trend to set up 
EQA systems that can be applied to both public and private providers 
alike. 

University and/or non-university sector institutions 

An EQA mechanism may relate to the university and/or 
non-university sector of higher education. Most commonly, 
EQA covers the university sector, because this is the segment of 
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higher education that used to have the highest degree of academic 
autonomy, in particular in the development of study programmes. 
Since non-university tertiary institutions, such as polytechnics, 
institutes or community colleges, were set up more recently, they 
were frequently put under the direct supervision of a governmental 
authority, which generally has power to create and supervise new 
study programmes. 

In some countries, however, EQA mechanisms address 
institutions from both the university and non-university sectors. The 
question then arises, however, whether the same methodology and 
set of criteria can apply to both types of institutions. Universities 
generally request that their study programmes be, when possible, 
at least informed by research. This is not necessarily the case in 
other tertiary level institutions, which are often of an applied nature 
and should offer training that is employment-oriented and convey 
practical competences. Some of the regional accreditation agencies 
in the United States have a specific commission that accredits 
universities and another that assesses community colleges (see 
Box 6). 



Box 6. Flexibility in the United States accrediting system 

Regional accrediting agencies have tried to maintain a single set of 
standards and rules while also acknowledging important differences 
in institutional type and mission. This tension has not been entirely 
settled, even today. At issue is whether expectations and requirements 
can be uniform across differing types of institutions, and whether 
all institutions need to provide comparable forms of evidence. This 
is pertinent especially to regional accreditation, where the entire 
institution is being evaluated. Even when it is acknowledged that there 
are important differences among institutions, difficult issues remain: 
what distinctions, and how many, are to be accommodated, and how 
distinctive can expectations be? 

Responses have taken different forms. Some accrediting agencies 
have created separate subunits for different types of institutions. For 
example, the Western Association of Schools and Colleges (WASC) has 
one accrediting commission to review community and junior colleges 
and another commission to review colleges and universities offering 
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a baccalaureate or higher degree. The two separate commissions 
establish their own standards and monitor policies for each type of 
institution. They are similar in many respects but different in others. 

Accrediting procedures make other adjustments to respect 
differences in institutional mission, allowing each institution to be 
judged in terms of its own chosen mission. A school of music, in this 
view, would be judged on different grounds than a school of engineering. 
Under this approach, the accrediting agency still examines whether a 
clear and coherently stated mission exists, whether there is evidence 
that this mission is being accomplished, and whether the institution 
has the resources necessary to be able to accomplish this mission in 
the near future. This approach, which had strong advocates during the 
1970s, is still found in the practices of regional accrediting agencies. 

Some intrinsic aspects of accreditation’s evaluation procedures 
lend flexibility. For example, present accreditation practice continues to 
look both to strengths and weaknesses of the institution. This approach 
gives flexibility because, even as evidence is assembled, there is room 
for applying discretionary judgement in the weighing of the evidence. 
Where certain areas are weak, the tradition of organizing evidence 
and reports that balance strengths and weaknesses serves to soften the 
impact of negative information as long as there are offsetting factors. 

Another approach makes small adjustments for institutional 
differences. Under this approach, the accrediting agency applies a 
single set of standards and criteria but, where possible in the accrediting 
review process, small adjustments are made to reflect institutional 
differences. For example, the accrediting review team may be composed 
of educators from similar institutions. A team sent to evaluate a small, 
relatively new college would not be made up of educators from the 
largest, most prestigious universities. A very prestigious university, 
in turn, would expect that its visiting team were made up of persons 
from similar institutional backgrounds. So, too, judgements about each 
institution are made in light of what can he expected for its size and 
relative resources. 

Source: El-Khawas, 2001: 49-51. 



Institutions and/or programmatic EQA 

Another basic option that all EQA systems have to address is 
the unit of analysis - that is, whether EQA should be institutional 
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or programmatic. EQA systems in institutions are naturally much 
broader than EQA programmes, and focus most frequently on 
the following domains of analysis: (a) mission; (b) governance; 
(c) effective management; (d) academic programmes; (e) teaching 
staff; (f) learning resources; (g) students and related services; 
(h) physical facilities; and (i) financial resources (Peace Eenn, 
2004 : 8 ). 

InstitutionalEQAinvestigates whether the missionand objectives 
of an HEI are appropriate, whether its resources and processes are 
appropriate to achieve them (under the fitness for purpose approach), 
or whether certain standards are attained. Institutional EQA looks at 
the institution as a system of which academic programmes are a part. 
Therefore, it needs to be relatively generic and pay less attention to 
the differences in objectives and performance among the different 
institutional subunits. Institutional EQA may be the preferred option 
in a system where quality varies widely between institutions, and 
when institutional management is rather weak. It may thus be a good 
means to strengthen the management capacity of an HEI. 

Programmatic EQA focuses on individual study programmes, 
many of which prepare students for a specific profession. Since each 
study programme may have its own policy on student recruitment, 
standards and criteria for curriculum, and also be subject to 
requirements arising from national qualification frameworks , it makes 
sense to assure the quality of individual programmes. Programmatic 
EQA may in particular assess whether an education programme 
is related to the professional expectations for entry into a specific 
profession. Over and above, institutions may offer programmes of 
varying quality in different disciplines that institutional EQA cannot 
recognize. Programmatic EQA is thus a strong tool by which to 
address issues of deficient quality at the level where improvement 
decisions have to be made - i.e. the department. 

However, it is also true that programmatic EQA must address 
many of the dimensions that relate to the broader institutional 
environment, such as the management of the institution, the 
department and the facilities that have a direct impact (either 
constraining or enabling) on the quality of the study programme. 
Therefore, any programmatic EQA necessarily also has an 
institutional dimension. 
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Both types of EQA are thus interwoven. Institutional EQA cannot 
he conducted without looking at programmes, hut programmatic 
EQA needs to look into the broader institutional environment. 
Countries usually start with a focus on either the institution or its 
programmes. However, it is eventually understood that both are 
complementary and nurture each other. Many systems that formerly 
had a clear focus on one specific aspect have decided with time to 
incorporate the other. Some countries conduct both and attempt to 
link them up in one single process. In some systems of EQA, such as 
that of the United States, both are carried out by different actors, but 
there is an attempt to co-ordinate the two so that they can enlighten 
each other. 



Box 7. Request for compulsory accreditation 

of state-approved programmes in Argentina 

In addition, the Law on Higher Education stipulates that state-regulated 
programmes have to be accredited by CONEAU in order to be offered; 
put differently, if an institution wishes to offer a state-regulated 
programme, it has to apply to CONEAU so that this agency assesses 
the programme to be offered. Like in the case of institutional 
accreditation, CONEAU’s assessment report is binding. Only courses 
with a favourable decision are entitled to issue official and qualifying 
degrees. The non state-regulated programmes need not have CONEAU’s 
accreditation; it is sufficient that they are authorized by the Ministry 
and fulfil the minimum requirements regarding the workload set by the 
aforementioned agreements for each case. 

The compulsory character of the accreditation of state-regulated 
programmes works retroactively since the programmes that were 
already offered also had to be submitted to CONEAU for assessment 
and accreditation. If they do not meet the standards and they obtain 
an unfavourable report from CONEAU, the Ministry is empowered to 
disqualify the programmes under consideration and waive the public 
recognition of the qualification. 

The accreditation processes for state-regulated programmes 
performed by CONEAU are carried out based on technical-academic 
standards set by the Ministry of Culture and Education and previous 
consultation with the University Council. They mainly aim at 
supplying an academic quality assessment, which should complement 



58 



International Institute for Educational Planning http://www.unesco.org/iiep 



Major organizational and methodological choices in EQA systems 



an institutional one. Even though the assessment’s goals are mainly 
academic, CONEAU’s decisions are binding for the Ministry 
of Education so that official recognition is granted to the new 
programme. 

The process of undergraduate programme accreditation comprises 
two steps: firstly, the implementation of a self-assessment on the part 
of the applying department, and secondly, the appraisal by a peers 
committee. 

According to each academic unit’s features, the self-assessment 
can take between one and four months. At its conclusion, a 
self-assessment report is issued, which has to provide systematized 
and comparable information, as well as a detailed appraisal of the 
conditions under which the programmes are issuing their results. This 
document may also include an improvements plan, which in the future 
allows compliance with the minimum standards set. 

Afterwards, the peer’s committee analyzes this self-assessment 
report together with other relevant information, pays a visit to the 
department, and makes a final decision, including the committee’s 
evaluative opinion and the recommendation for temporary accreditation 
or for rejection of the application. 

Both the self-assessment and the peer’s committee appraisal 
stages are carried out following certain methodological instruments, 
designed for each purpose. The Self-Assessment Guide was designed to 
organize and co-ordinate the academic unit’s self-assessment task. The 
Peers Guide outlines a logical appraisal sequence, laid out in sections 
and cores, which enables the assessment of the programme’s current 
state, to link from the outlook of undergraduate education its features 
to one another and to the practices that the academic community has 
adopted, and to check standard compliance. 

Source'. Villanueva, 2007. 



All programmes? Or only certain types of programmes 
(for instance state-approved)? 

When EQA is programmatic, some countries chose to focus 
on certain types of programmes, for instance teacher education, 
medicine or engineering, which prepare for professions that are 
perceived to he vital for a country. In Argentina, for instance, it 
is the function of the national accreditation agency, the National 
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Evaluation and Accreditation Council (CONEAU), to accredit 
programmes that require state approval. This is related to the fact 
that traditionally it is the institutions that created and managed their 
study programmes. However, over time, the state considered that 
those study programmes that lead to professions of public interest 
were in need of tighter public regulation. 

Use of quantitative versus qualitative data 
for accreditation 

EQA systems, standards-based or not, most commonly use 
quantitative data combined with qualitative judgement. Most quality 
assurance agencies have developed instruments that may either 
consist of open questions to focus on qualitative analysis, or request 
the collection of a set of statistics. Peer review is, however, typically 
a phase where qualitative judgement is the prevailing mode. 

A request for quantitative data refers to input factors (such 
as final secondary school grades of incoming students, teachers’ 
qualifications, financial resources), throughputs (e.g. student:staff 
ratios) or outputs (e.g. number of graduates and their average grades, 
number of publications or patents). EQA puts data in an evaluative 
framework, which is either available at the institution by means of 
statistics collected periodically or for the special purpose of quality 
steering (such as tracer studies, alunmi’s and employers’ opinions). 

Quantitative data allows comparison of performance either 
among departments or institutions. However, many problems exist 
with regard to the interpretation of these indicators, such as how 
to judge data on unit cost per student, and whether high unit cost 
means good quality or rather wastage of resources. In addition, 
it has been argued that the use of indicators, if for comparison 
purposes, may lead to standardizing behaviour, in particular when 
public funding is linked with specific expected outcomes measured 
through indicators. This could impinge, it is argued, on the creativity 
of academic activities and their outcomes. 

In addition, many standard statements are of a qualitative nature 
and cannot easily be assessed through quantitative data. A statement 
such as “the institution has a clear and comprehensible statement that 
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guides it, is approved by the governing board, and communicated to 
the institution’s constituencies” (SACS Commission on Colleges, 
1998) cannot easily be assessed through quantitative data, and 
requires qualitative analysis and judgement made by a peer with 
an appropriate set of references based on a series of internal 
interviews. 



Box 8. Assessing a criteria-based quality model in India 

During the on-site visit, keeping in mind the philosophy of NAAC 
[National Assessment and Accreditation Council], the peer team 
does an objective assessment of the quality of education offered in 
the institution through three major activities - visiting departments 
and facilities, interacting with various constituencies of the institution 
and checking documentary evidence. The interaction with the faculty 
normally happens when the team goes to departments and facilities. 
Separate sessions are arranged for interaction with management, 
administrative staff, and representative groups of students, parents and 
past pupils. It has been found that interactions enhance the team’s ‘feel’ 
of the institution. For each interaction NAAC has evolved a guiding 
agenda ... 

The interaction with the representative group of students has 
proved to be a very fraitful part of the visit. To manage the interaction 
effectively, it is generally restricted to around 50 students. However, 
the questionnaire evolved by NAAC on the campus life of students is 
distributed to a cross-section of students at random, managed directly 
by the NAAC officer, and this input is also made available to the 
peer team before it meets the representative group. The confidential 
feedback from students often helps the team identify areas that need 
more attention. 

The application of weightages 

Validation and subsequent assessment lead to an overall 
institutional score and a detailed report, which begin with the team 
agreeing on criterion-wise scores. Taking cognizance of the variance 
in types of institutions, different criteria have been allotted differential 
weightages ... The institutional score is further used by NAAC to obtain 
the overall grade. Nevertheless, the role of the peers is only to give the 
institutional score to NAAC with the detailed report. 

Source: Stella, 2003: 83-85. 
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Whatever the comhination of quantitative and qualitative data 
used in the quality assurance process, human judgement is always 
applied to both quantitative and qualitative information. However, 
EQA systems may opt to provide a quantitative framework for 
qualitative judgement hy applying quantitative indicators to 
qualitative dimensions and attaching scores to them. This option is 
frequently adopted in newly established accreditation systems where 
there is a particular need for transparency and perceived ‘objectivity’ . 
In systems with a tradition of accreditation and/or where there is a 
corps of professional assessors, one finds more reliance on human 
judgement. Such methodological options in accreditation are very 
important for the credibility of the process. 

Table 2.1 presents a schematic relationship of the overall 
purposes of EQA systems and major organizational choices. While 
a combination of the three major purposes is possible, it is generally 
understood that any system will be predominantly geared towards 
one purpose or another. 



Table 2.1 Classification of basic options used in EQA systems 



Purpose 


Quality control 


Accountability/ 
public assurance 


Improvement/ 

guidance 


Preferred 

mechanism 


Licensing 


Accreditation/ 

assessment 




Framework for QA 


Standards-based 

approach 


Fitness for purpose 
+ fitness of purpose 


Fitness for purpose 


Procedures 


Mostly external 
assessment 


Both external and 
internal assessment 


Mostly self- 
assessment 


Nature 


Compulsory 


Compulsory or 
voluntary 


Voluntary 
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The basic three-stage process 

While there are various approaches to quality assurance which 
correspond to unique national contexts, quality assurance agencies 
generally follow a three-stage process, which will he discussed in this 
chapter. First, however, some common features are given helow: 

• They base assessment on predetermined and transparent 
criteria. 

• They use a combination of self-assessment and external 
review. 

• The extent of the evaluative nature of the self-assessment may 
vary. Many of the specialized or professional accreditors merely 
ask for information without requiring analysis or evaluative 
judgements on the part of the programmes being assessed. 

• They emphasize public disclosure of the outcome (although the 
extent of public disclosure varies from disclosure of only the 
final outcome to disclosure of the full assessment report). 

• They ensure validity of the assessment outcome for a specific 
period. 

Preparation of a self-assessment report by the institution/ 
programme 

The first stage of the three-stage process consists of provision 
by the institution (or programme) of the relevant information related 
to predetermined, well-publicized criteria. In most cases, this is 
also accompanied by a self-assessment, which provides a critical 
analysis of the information. Self-assessment is the central element 
in most EQA procedures. All quality assurance agencies emphasize 
and recognize the value of an analytical and self-critical process, 
but many also recognize that it is not always practical or realistic 
to expect higher education institutions to carry them out for the 
following two reasons: 

• In the absence of a ‘culture of evaluation’, self-assessment may 
be uncritical; in particular when the stakes are high (e.g. when 
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quality assurance procedures may lead to sanctions, or approval 
is essential for continuing operation of the programme or the 
institution), it is unrealistic to expect institutions to carry out a 
truly critical analysis. 

• When agencies operate internationally, they tend to prefer the 
provision of data and take care of the evaluation themselves. 

A set of standards and criteria determined hy the quality 
assurance agency forms the basis for the self-assessment. The 
agencies generally carry out national consultations and ensure wide 
participation of the stakeholders in developing the standards and 
criteria. There may he variations in the method adopted: For example, 
some agencies might apply only the predetermined criteria to all 
of the institutions and programmes (the standards-hased approach), 
whereas others might carry out the review against an institution’s 
own goals and objectives (the fitness-for-purpose approach), and 
the rest might take an intermediate approach. Flowever, the place 
given to an institution’s own goals and the standards set by the 
quality assurance agency are made clear to the stakeholders prior 
to applying the quality assurance procedures. The institution (or 
programme) undergoing the process is asked to do a self-assessment 
and report on how it meets the standards set, or criteria identified, by 
the quality assurance agency. 

In the self-assessment process, academics (together with 
administrators within the department/institution, based on a catalogue 
of either open questions or indicators to be collected) discuss the 
strengths and weaknesses in their respective units, and identify the 
causes of possible shortcomings. They usually decide for themselves 
on the strategies to be used to attempt to improve quality. This latter 
methodology has the advantage of directly involving competent 
professionals in the improvement process: those who will be in 
charge of implementing reform action. Also, it helps set up a culture 
concerned with quality in the long term, and it might strengthen 
community spirit, which is often lacking in academia. 

Despite the difficulties mentioned earlier, the underlying 
assumption in insisting on self-assessment is that an institution 
that really understands itself - its strengths and weaknesses, its 
potential and limitations - is likely to be more successful in carrying 
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out its educational mandate than one without such self-awareness. 
Self-assessment is thus envisaged as the hackhone of the process of 
quality assurance. It is through the self-assessment report that the 
external review team tries to understand and tentatively evaluate the 
institution or programme prior to the site visit. 

External review 

The second stage is a site visit of an external review team to 
validate the self-assessment or the institutional report that results 
in the report/recommendation to the quality assurance agency on 
the quality of the institution/programme. Like self-assessment, 
this stage has evolved as an internationally accepted component of 
quality assurance. The expert taking part in the quality assurance 
process is referred to as the external reviewer, who is knowledgeable 
in the language, categories, rationale and codes that belong to the 
discipline or the profession of the programme or institution being 
assessed, and therefore is a peer to the people being reviewed. At the 
same time, the external reviewer comes from outside the programme 
or institution, and therefore provides an outsider’s perspective. When 
the institution submits the self-assessment report, a team of external 
reviewers constituted by the quality assurance agency analyzes the 
report of the institution and validates the claims made, generally 
by visiting the institution. The visit by the review team gives the 
institution an opportunity to discuss and find ways of consolidating 
and improving the academic environment. 

The external review is expected to provide an outside perspective 
and often to validate the conclusions drawn from the self-study; it 
uses professional judgement (national or international subject matter 
experts or professionals, etc.); it collects information by means of 
site visits and personal interviews with internal, and sometimes 
external, stakeholders of the higher education institution. It is thus 
able to take account of the conditions under which certain results 
could be attained. 

Decision-making and reporting the outcome 

Based on the report of the institution or programme and the 
recommendations of the review team, the agency, in the third and 
last stage of the quality-assurance process, either makes the final 
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decision or gives advice to a public authority (such as the ministry of 
education) who will make the final decision. In all quality assurance 
mechanisms, there is an element of public disclosure of the outcome, 
although the extent of public disclosure varies. It may vary from 
disclosure of only the final outcome, as in the case of a typical 
accreditation, to disclosure of the full assessment report as in the 
case of a typical audit. In general, in systems where the report is the 
only outcome, it is made public. In systems where a formal decision 
on accreditation status is made and a report written, the extent of 
public disclosure also varies. The outcome is generally valid for 5 to 
10 years. 

Decision-making may either warrant a simple ‘yes’ or ‘no’ 
decision, or it might require elaboration, for example ‘based on 
certain conditions ...’ Frequently, accreditation also involves a 
supplementary grading system that is an add-on to a simple ‘yes’ 
or ‘no’ decision. It is common practice then to publish the decision 
made, with or without the expert report prepared by the peer team. 



Box 9. Adoption of the three-stage model of EQA in India 

In India, the explicit focus on external quality assurance in higher 
education is of recent origin. In the 1980s, it was felt that the 
unprecedented expansion of higher education in India during the 
previous fifty years had rendered the built-in regulatory mechanisms 
inadequate. There were criticisms that the country had permitted 
the mushrooming of institutions of higher education with fancy 
programmes and substandard facilities, and consequent dilution 
of standards. To address the issues of deterioration in quality, the 
National Policy on Education (1986) advocated the establishment of an 
independent national accreditation body. Consequently, the University 
Grants Commission (UGC) established the National Assessment and 
Accreditation Council (NAAC) as an autonomous body in September 
1994. 

Although the quality assurance experience of India seems to 
be just ten years old, it should be seen against the backdrop of the 
quality controls that the Indian higher education system has had for 
the past 150 years. With regulatory and recognition mechanisms 
already in place, the objective of national accreditation is to lead 
institutions of higher education towards maximizing their potential for 
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quality education, contrary to the minimum standards ensured by the 
regulatory mechanisms. This makes the Indian system of accreditation 
unique in many ways. 

The limited resources available for improvement in quality of 
higher education, and the size and complexity of the higher education 
system in India, are other factors contributing to India’s unique 
scenario of accreditation. With the third largest higher education 
system in the world - 322 university level institutions and more than 
16,000 colleges catering for 9 million direct and full-time students 
- developing a national quality assurance mechanism and making the 
process operational have been formidable tasks. 

Since its inception in September 1994, the NAAC spent the first 
three to four years evolving its policies, principles and instruments. 
An analysis of the current practices of accrediting agencies of various 
countries reveals that most quality assurance systems have certain 
common elements - self-evaluation and peer review - and NAAC 
adopted these core elements. To address the contextual considerations, 
the NAAC took a clear line in addressing aspects such as; its role in 
assessment (NAAC does not take a direct role in assessment), the 
nature of the assessment process (it is a voluntary process), the focus 
of assessment (improvement is the focus, contrary to accountability 
concerns observed in many countries), the linking of the assessment 
outcome to funding (not linked to basic funding), the unit of assessment 
(the institution is the unit of assessment), the policy on disclosure of the 
assessment report (a full assessment report and the institutional grade 
are made public), and the period of validity (five years). The way the 
NAAC firmed up its rationale and stand on these issues would be very 
useful to emerging quality assurance agencies. With this assessment 
model, a lot more emerged at the practical level when the assessment 
efforts proceeded. 

Source: Stella, 2004: 9. 



The three-stage model as a basic choice in EQA has now 
reached wide consensus in the quality assurance community, hut 
it is a cumhersome and costly approach. When quality assurance 
focuses on higher education programmes in relatively hig systems, 
it becomes increasingly important to look for alternative, lighter and 
cheaper models of EQA. Such alternative models may reduce the 
emphasis on any one of the phases: either self-study (which could 
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simply be based on tbe provision of statistics), peer review (which 
could be conducted perhaps at a distance), or simply drop one of 
the stages altogether. Creative thinking in this respect will become 
increasingly important if EQA systems are expected to cover all 
higher education programmes in systems of a certain size. 

Areas of quality assessment 

In addition to the three-stage model of EQA, the areas or aspects 
considered by quality assurance agencies have much in common. 
In fact, while they may have different names, or follow different 
organizational structures, most quality assurance agencies look at 
the same things, albeit with different emphases. Eor example, in 
the Philippines there are four quality assurance agencies that do 
programme accreditation. The table in Box 10 highlights how similar 
they are in the scope of their quality assurance. The areas that are 
considered by the quality assurance agencies that do institutional 
accreditation are also similar. 



Box 10. Standards for quality assurance in the Philippines 

Programme accreditation hy the four accrediting associations 
of the Philippines 

The (accreditation or quality assurance) agencies engage qualified 
faculty members and professionals to develop detailed criteria specific 
to each programme or course of study. The criteria may differ from 
one agency to another, as might their application, but the scope of the 
review based on the areas covered by the standards of each agency is 
almost identical. 



S. No. ACSC-AA 


PAASCU 


PACU-COA 


AACCUP 


1 Purposes and 


Purposes and 


Purposes and 


Mission, goals 


objectives 


objectives 


objectives 


and objectives 


2 Faculty 


Faculty 


Faculty 


Faculty 


3 Instruction 


Instruction 


Instruction 


Curriculum 








and programme 
studies 


4 Library 


Library 


Library 


Library 
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5 


Laboratories 


Laboratories 


Laboratories 


Physical 
facilities and 
laboratories 


6 


Physical plant 
and facilities 


Physical plant 
and facilities 


Physical plant 
and facilities 




7 


Student 

personnel 

services 


Student 

services 


Student 

personnel 

services 


Students 


8 


Social 
orientation 
and community 
involvement 


Social 
orientation 
and community 
involvement 


Social 

orientation and 

community 

involvement 


Extension and 
community 


9 


Organization 
and research 
administration 


Administration 


Organization 

and 

administration 


Administration 



Legend: ACSC-AA = Association of Christian Schools and Colleges Accreditation 
Agency 

PAASCU = Philippine Accrediting Association of Schools, Colleges and Universities 
PACU-COA = Philippine Association of Colleges and Universities Commission on 
Accreditation 

AACCUP = Accrediting Association of Chartered Colleges and Universities in the 
Philippines 

Source'. Phelps in Arcelo, 2003: 72-73. 



Even among agencies that differ in terms of the country context 
in which they operate, the unit of quality assurance etc., there is 
agreement on the areas that are key to assessing quality. 

The ahove-mentioned areas are only indicative of how a group of 
quality assurance agencies has identified key areas that have a hearing 
on the quality of institutions. Some of them are expressed in terms of 
quantity, and some would require a qualitative assessment. The case 
of the Philippines given in Box 10 highlights programme quality, 
showing that areas of assessment overlap for both institutional and 
programme accreditation, although there may he differences in terms 
of focus and scope. While the curricular aspects under institutional 
accreditation may concern mainly the overall policies and practices 
of the institution, the programme accreditation would look more 
closely into the quality of the curriculum of the programme under 

69 



International Institute for Educational Planning http://www.unesco.org/iiep 



External quality assurance in higher education: 
making choices 



review. The institutional accreditation might also look at the quality 
of one or more programmes to seek evidence for the evaluations; hut 
the purpose is not to pass judgement on the quality of the curriculum 
of that programme. 

Approaches to quality assessment 

Based on a specific definition of quality, quality assurance 
agencies huild their frameworks for translating it into ‘quality 
assurance decisions ’ . A critical element in quality assurance is the use 
of evaluative guidelines or frameworks against which the agency can 
make decisions. The quality assurance process may examine many 
academic and administrative aspects of the institution or programme 
being reviewed and collect data on those aspects. However, the 
information gathered does not speak for itself; an evaluative 
judgement must he made, and the evidence that is gathered must he 
interpreted in the light of some prior questions. This may often he done 
in a rather explicit fashion, where both quantitative and qualitative 
benchmarks are set for desirable achievements and the assessor 
simply establishes the evidence. However, there are also systems 
where the assessment is based on the professional judgement of the 
assessor. This use of evidence, judged against a quality assurance 
framework, leads to decisions that have important consequences. 
Agencies do it in many ways - some develop standards, some agree 
on a set of indicators, and some define benchmarks. While some 
agencies develop specific indicators, there are agencies that develop 
broad standard statements against which quality is assessed by 
means of expert judgement. 

The agencies develop and use standards in different ways. The 
standards prescribed by the All India Council of Technical Education 
(AICTE), and given in Box 14, are mostly about inputs required in 
the institution to offer a programme of quality. Increasingly, however, 
agencies tend to shift their focus to outcomes. In most programme 
accreditation systems in professional areas of studies, standards relate 
to good institutional procedures and practices with a professional 
perspective. These agencies have interpreted quality in terms of how 
effectively new entrants to a certain profession have been prepared 
for their responsibilities. In recent years, this has resulted in many 
professional bodies paying attention to competency-based standards 
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in order to evaluate quality. Competency-based standards focus 
on the appropriate and effective application of knowledge, skills 
and attitudes. They emphasize the relationship between formal 
education and work skills - the ability to apply relevant knowledge 
appropriately and effectively in the profession. The agencies that 
adopt this understanding of quality generally require institutions 
and programmes to demonstrate the output of the programme 
rather than the input; i.e. on developing expertise among students to 
become competent professionals rather than on the number of hours 
of tutorials or hands-on experience provided. Box 11 shows how 
competency-based standards were developed in the United States. 



Box 11. Move towards competency-based standards of 
professional bodies (United States) 

The evolution of standards for programmes in architecture provides 
an illustration (National Architecture Accrediting Board, Conditions 
and Procedures, 1991, pp. 4-5). As early as 1902, following the 
precedents established in law and medicine, practitioner groups had 
developed an examination system in Illinois for graduates of fourth- 
year programmes in architecture. By 1914, minimum standards for 
architecture programmes were established. In 1940, a national board 
was created in order to oversee accreditation of schools of architecture 
on a national basis. While numerous revisions of this basic approach 
occurred over the next several decades, a significant new approach was 
adopted in 1982. The board’s new mandate was to apply “achievement- 
oriented performance criteria” in its evaluation of architecture 
programmes. Under this approach, each school “... is responsible 
for seeing that each graduate completes a liberal studies requirement 
and attains the necessary achievement for each of the ... major areas” 
of the programme. Criteria are grouped under four major headings: 
fundamental knowledge; design; communication; and practice. Levels 
of accomplishment are stipulated for 54 different areas of practice. 
Source: El-Khawas, 2001: 63-64. 



The bodies that regulate the practice of the profession 
develop their methodologies based on the competency-based 
standards in many ways. For example, the Canadian Institute 
of Chartered Accountants (CICA) has developed the ‘The CA 
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candidate’s competency map’ for its qualification (recognition 
or registration) process of chartered accountants. CICA, together 
with the chartered accountant institutes, represents approximately 
68,000 chartered accountants and 8,000 students in Canada and 
Bermuda. It has identified two types of competencies - pervasive 
qualities and skills (that all chartered accountants are expected to 
demonstrate in all tasks), and specific competencies. The specific 
competencies are grouped into six categories. To cite an example, 
the competencies listed hy CICA for one of the categories, namely 
‘Taxation’ (competencies related to taxation planning, compliance 
and reporting for various entities), are given in Box 12. 



Box 12. The competency map (Canada) 

The specific competencies 

Taxation 

1. Analyzes the entity’s tax profile and identifies overall tax issues 

1 . 1 Seeks to improve the entity’s tax profile 

1.2 Evaluates and advises management on applicable new tax 
legislation on an ongoing basis 

1.3 Identifies, analyzes, and advises on compliance and filing 
requirements 

1.4 Analyzes the range of professional expertise required to 
advise on potential tax issues 

2. Prepares and files necessary returns in accordance with legal 

requirements 

2. 1 Advises on tax compliance 

2.2 Meets filing requirements 

3. Practices effective tax planning to maximise after-tax returns 

3.1 Identifies, analyzes, and advises on specific tax planning 
opportunities 

3.2 Analyzes tax consequences of transactions and business 
opportunities 

4. Supports, defends, and negotiates tax positions 

4. 1 Analyzes and responds to assessments 

4.2 Prepares information to support objections, appeals, and 
court litigation 

Source: www.cica.ca 
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The examples above indicate that quality assurance agencies 
tend to choose their broad approach considering many contextual 
factors from among the various options available. The approaches 
allow for varying levels of professional judgement. Most quality 
assurance agencies have some level of specifications and reliance 
on quantifications. In some quality assurance frameworks, the peers 
have more freedom to make judgements against a broad framework; 
in most other systems there are specifics that guide peer judgement, 
such as quantitative specifications and indicators. 

Reliance on quantitative assessment 

Quality assurance agencies may rely on quantification at various 
levels by requiring that institutions demonstrate that they comply 
with certain quantitative norms; that peers assess whether the norms 
are met; that the peer assessment be recorded on a quantitative scale; 
and that the final outcome be expressed on a quantitative scale. This 
raises the frequently asked question of whether quality should be 
assessed against quantitative measures. 

When quality relates to consistency, compliance or agreement 
with expected levels of performance, quality assurance agencies 
tend to develop quantitative norms and use them as the frame of 
reference for quality assurance, such as the AICTE standards 
mentioned earlier. Agencies that do rely on quantification tend to 
look into the excellence of institutions. For example, the National 
Council of Accreditation in Colombia (NCAC) focuses on excellence 
and defines quality as the integration of 66 characteristics. For each 
characteristic, a series of qualitative and quantitative variables are 
spelt out. Box 13 highlights how the variables and indicators have 
been spelt out for one of the characteristics. 

Irrespective of whether the agency seeks to ensure minimum 
standards or standards of high quality, reliance on quantification 
can be found. The Colombian example demonstrates how quality 
in one specific aspect can be affected by a number of indicators. It 
also shows how assessing quality is a complex task that needs to 
be balanced with peer assessment. However, there are agencies that 
base their decisions mostly on quantitative data. 
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Box 13. Variables and indicators of a characteristic 
(Colombia) 

Characteristic 16: In compliance with institutional objectives and 
relevant programme specificities, faculty size is adequate and teachers 
have both the commitment and training the programme requires. 

Description: It points to the fact that, to achieve the institution 
and programme objectives, the required number of teachers should be 
available, their level of qualification appropriate and their commitment 
to the institution and to the programme in question adequate. Likewise, 
efforts are made to find out whether the number of teachers attached 
to the programme and their training and commitment come close 
to the ideal situation sought after for the specific programme and 
institution. The above examines the quality of education in one of its 
core aspects. 

Variables: 

Adequacy to programme requirements of faculty commitment 
and of their specific training and level of qualification. 

Academic quality of faculty attached to the programme. 
Indicators: 

1) Training (graduate, postgraduate. Master’s, Doctoral), rating on 
the promotion ladder and commitment of teachers to institution 
and programme. 

2) Other educational experiences of teachers relevant to their 
performance in the programme. 

3) Period of time teachers have worked in the institution and 
programme, as well as any other academic and professional 
experiences of faculty involved. 

4) Relationship between the number of students enrolled in a 
programme and the number of teachers involved. A comparison 
should be established with regard to full-time commitment. 

5) Assessment by outstanding members of academic communities 
of faculty committed to programme. 

6) Assessment of programme students with regard to both the quality 
and sufficiency of the number of students enrolled, and of the 
commitment of teachers involved in the programme. 

Source: Revelo and Hernandez, 2003: 47-48. 
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Quantification to guide peer assessment 

Reviewers may be required to follow certain guidelines related 
to quantitative measures, within which the qualitative judgement 
has to be made. For example, although oriented towards peer 
assessment, the accreditation methodology of the National board of 
Accreditation (NBA) in India, as depicted in Box 14, requires the 
reviewers to score each indicator, the maximum for each indicator 
being pre-determined by NBA. 



Box 14. Quantification to guide peer assessment: 

NBA (India) 

Each of the eight criteria has been broken down into parameters, and 
weightages have been assigned to these parameters by the NBA. The 
parameters and the weightages assigned to them, which are different 
for diploma, undergraduate (UG) degree and postgraduate (PG) degree 
programmes are given below: 

Parameters Marks 

Diploma Undergraduate Postgraduate 

I. Organization (30) (80) (50) 

and governance 

A Planning 

and monitoring 
B Recruitment procedure 
and its effectiveness 
C Promotional policies 
/procedure 
D Leadership 
E Motivational initiatives 
F Transparency 
G Decentralization 
and delegation and 
participation of faculty 
H Constitution of governing 
council/governing board 

(Note: The indicators will be weighted.) 

Source: www.nba-aicte.ernet.in/parameter.doc 
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Quantification in reporting outcome 

To cite an example of how reporting outcomes may be 
quantified, in the case of NAAC, the scores given by the reviewers 
are further used to calculate the institutional scores in percentage. 
The institutional score determines the institutional grade on a 
nine -point scale: Grade C denotes the score range 55-60, C-H denotes 
60-65, C++ denotes 65-70, B is 70-75, B+ is 75-80, B++ is 80-85, 
A is 85-90, A-h is 90-95, and A-r-i- is 95-100. Institutions that do not 
achieve the minimum 55 per cent are not accredited. 

Some more recent systems follow this approach to establish 
credibility (especially in the absence of a well-established corps 
of assessors or in big systems with a lot of inter-team variance) 
and ensure objectivity in their accreditation decisions. However, 
the relationship between numbers and objectivity is questionable. 
Numbers help only when certain assumptions can be made; that is, 
when you can be sure that the difference between 50 per cent and 
60 per cent is the same as the difference between 75 per cent and 
85 per cent, for example. In practice, this is not usually the case, and 
quantitative measures give a misleading sense of objectivity, hiding 
the real subjectivity involved in setting the scores. 

Reliance on quantification has been debated by different 
stakeholders for various reasons. For example, it may help an agency to 
ensure consistency in its approach, and minimize inter-team variance 
among the review panels. It might also be very useful in emerging 
systems to assure transparency. However, it may encourage higher 
education institutions to report simple quantitative measures that 
benefit them instead of truthful qualitative assessments. Fears have 
also been expressed regarding the relevance, accuracy and efficacy 
of many measures that have been, or are likely to be, employed 
by the quality assurance agencies. Reliance on quantification and 
quantitative indicators becomes most controversial when the 
emphasis shifts from their use as one of several inputs of effective 
decision-making by the review team to using them as a ranking 
device. Much depends on how reliance on quantifications is balanced 
with peer assessment. 
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Reliance on professional judgement 

Many quality assurance agencies therefore do notprovide explicit 
norms and quantitative targets because once they are made explicit 
they might become counterproductive to institutional diversity and 
the fitness for purpose approach. It does not mean that compliance to 
standards is not important, but there may be other mechanisms that 
ensure complianee. Once a threshold level is already ensured, the 
ageney cheeks how well the HEIs are performing in their own way 
to achieve their goals and objectives. Here, considering diversity 
is important and reliance on quantitative assessment may not be of 
help. Professional judgement that adheres to the agency’s quality 
assurance framework is central. 



Box 15. Indicative scope of AUQA (Australia) 

AUQA pays particular attention to the academic activities carried out 

in the institution’s name. Indicative scope of an institutional audit 

includes: 

• Organizational leadership and governance, planning. 

• Teaching and learning (all modes); processes for programme 
approval and monitoring; comparability of academic standards in 
on-shore and offshore programs. 

• Research activities and outputs, including commercialization. 

• Community service activities. 

• Internationalization, including contracts with overseas partners. 

• Support mechanisms for staff and students. 

• Communication with internal and external stakeholders. 

• Systematic internally-initiated reviews (e.g. of departments, 
themes), including the rigour and effectiveness of the review 
mechanisms employed. 

• Administrative support and infrastructure. 

Source: Audit Manual: www.auqa.edu.au 



Agencies that do not want to be very prescriptive do not set 
specific quantitative targets for institutions to comply with. However, 
they may provide detailed guidelines (or standards) on issues such 
as demonstrating adequacy and efficiency. For example, an agency 
may not insist that for every 10 students there should be a teacher. It 
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might not insist that the postgraduate programmes he handled only 
hy the doctoral degree holders; however, it might say in general 
language that it should have adequate and competent faculty to run 
the programme under review. 

For example, the Australian Universities Quality Agency 
(AUQA) gives only the indicative areas to he covered and underlines 
professional judgement of the peers. 

Obviously, such a hroad definition of quality concern widens 
the scope of subjectivity since agencies rely largely on professional 
judgement. The quality assurance agencies handle this concern 
by developing manuals and guidelines to guide peer assessment. 
A rigorous training strategy is the key to ensuring reliable peer 
assessment. An interesting strategy that helps enhance the objectivity 
of a peer review team’s judgements is the requirement that they 
reach their conclusions by consensus, not by vote. Thus, objectivity 
is ensured through a measure of inter-subjectivity, as extreme views 
are dismissed and what prevails is what all members of the team 
agree on. The composition of teams and the way they cover different 
views and disciplinary approaches are also essential to making sound 
decisions. 

In general, as the discussions above have revealed, quality 
assurance agencies are found to rely on both quantification and peer 
assessment. They have to choose an appropriate stand to suit the 
context and their mandate. The approaches discussed in this chapter 
are not necessarily to be taken as measures to be implemented in 
isolation, but rather as approaches that can be used in combination, 
as they each have different strengths and weaknesses. This is 
particularly true for the quantitative/qualitative debate. 
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In most countries there is an agency that is responsible for defining 
the scope of quality assurance, elaborating the methodology - usually 
in consultation with academia and stakeholders - and conducting the 
process involving the external reviewers. It also prepares guidelines 
and handbooks for all those involved in the process, and offers 
workshops for HEIs and training for reviewers. It carries out these 
functions with a small core staff and relies on external reviewers 
generally drawn from the academic community. Depending on 
factors such as the size of the higher education system to be covered 
by the quality assurance agency, and the level of involvement in the 
quality assurance process, the agency requires human and financial 
resources to support its functions. 

Quality assurance agencies may be established by the 
government, HEIs, or private groups. Except for a few agencies 
owned by the HEIs themselves, or which have significant support from 
the HEIs, most of the recent quality assurance agencies have been 
governmental initiatives and clearly serve governmental functions. 
In some countries, professional accreditation has developed as a 
mechanism independent of the government and HEIs. Irrespective 
of the affiliation of the agency, even for government initiatives, its 
autonomous/independent nature with respect to quality assurance 
decisions is seen as desirable. Careful consideration of the national 
context in terms of size of the system to be covered, scope of the 
quality assurance and level of involvement of the quality assurance 
agency is necessary. 

Various safeguards and protocols are followed to establish 
the objectivity and reliability of the quality assurance process and 
its outcome and thereby assure the credibility of the agency to the 
stakeholders. In general, quality assurance agencies are accountable 
to one or more major higher education stakeholders - governments, 
HEIs, the academic community and the public at large. Depending on 
the institutional affiliation, there are built-in mechanisms that make 
the quality assurance agency accountable to its governing body. 
There are also various measures that ensure the accountability of the 
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agency, such as joining regional or international quality assurance 
networks and undergoing meta-evaluations. 

Administrative affiliation of the quality assurance 
agency 

The administrative affiliation of the quality assurance agency has 
certain implications for its autonomy. Although most of the quality 
assurance agencies - including those established and funded hy their 
governments - claim some level of autonomy from government, 
obviously the non-govemmental bodies can claim the greatest 
independence in decision-making. It often happens, however, that 
a government official, such as a representative from the education 
ministry, either sits in on or chairs the agency’s governing body. 
When owned by the HEIs, quality assurance depends on the voluntary 
acceptance of the procedures by the member institutions, and the 
HEIs shape the nature and the framework of the quality assurance 
process. This ‘bottom-up’ orientation is found in the United States 
where accreditation agencies are non-governmental agencies of 
HEIs. In this case, the issue is: What level of independence can the 
agency claim with regard to the HEI itself? 

The affiliation of the quality assurance agency is a much- 
debated issue in relation to the purpose of quality assurance. 
The issue is highly political and ideological. For some groups, 
government affiliation is seen as an external and bureaucratic 
approach where quality is controlled, while ownership by an HEI 
is seen as an internal, non-bureaucratic approach with a focus on 
quality improvement rather than control. However, there is no 
simple or direct relationship between ownership of the quality 
assurance agency and the balance between quality improvement 
and control. Many government-owned systems emphasize quality 
improvement, and some institution-owned agencies tend to act as 
gatekeepers, preventing the entrance of newcomers to the higher 
education market. The objective and focus tend to be independent 
of ownership. 

Getting the government to support the quality assurance 
process without losing any of the agency’s autonomy or affecting 
its functioning is certainly an option to be considered. In countries 
where the higher education system itself is undergoing reform. 
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quality assurance initiatives with considerable autonomy are being 
developed by the government as a part of the reform strategy. In 
more mature systems, the HEIs may take a leading role by providing 
external reviewers, or by taking part in different stages of the 
process. They may thus be in a position to shape the important 
developments of the quality assurance system. In the United States, 
and later in the Philippines, institutional accreditation has evolved 
as a process shaped by the HEIs themselves, but in most other 
systems the initiative has been taken by the government. In any case, 
government support in using the quality assurance outcome for vital 
decision-making, such as funding incentives, strengthens the quality 
assurance processes. 

There are also organizations that are established and managed 
by groups other than HEIs and governments. This is how 
accreditation of programmes in professional areas of study evolved 
and is still practised. Known as ‘specialized accreditation’, this 
type of accreditation was born out of the concern of a profession 
about the quality and relevance of the educational programmes that 
were preparing its practitioners, and the quality of the practitioners 
trained. Specialty councils or professional bodies carry out this type 
of quality assurance through licensing or registration procedures. 
The main assessment criterion of these agencies is the quality 
of the graduates - future practitioners of a profession - and the 
procedures are developed and monitored by current practitioners of 
standing. Protecting public interest and safeguarding the standards 
of professional practice are central to these agencies. The quality 
assurance decisions of these agencies have implications for 
practitioners at national and international levels. 

Some professions, like medicine, are highly protected, and 
they have tightly regulated practices. Consequently, most countries 
have some amount of well-regulated quality assurance practices 
for professional areas of studies. In some countries, a degree in a 
certain discipline qualifies the graduates to practise as professionals 
in that field, and in such cases those programme offerings are mostly 
regulated by professional bodies. 

Thus, a quality assurance agency can be established with 
greater or lesser affiliation to the government. There are four levels 
of affiliation possible: 
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• It can be established as a governmental agency, perhaps as a 
unit in the ministry; 

• It can be a body fully independent of the government, 
without any governmental intervention in its establishment or 
functioning. An example of this type of structure is a quality 
assurance agency established by HEIs; 

• It may be a buffer body or established under a local buffer 
organization where the government plays a role in its initiation. 
In this instance, it will serve governmental functions, but will 
not necessarily be under government control; 

• It can be a body established without any intervention of 
government or the HEIs in its establishment or functioning. 
Professional accreditation is a typical example (professional 
councils). 



Box 16. Association of Professional Engineers of Nova 
Scotia (Canada) 

The Association of Professional Engineers of Nova Scotia (APENS) is 
the licensing and regulatory body for the more than 4,500 professional 
engineers and engineers-in-training practising in Nova Scotia or on 
Nova Scotia Projects. 

APENS mission 

• To serve and protect the public interest. 

• To advance and promote the value and proficiency of the 
Engineering Profession. 

• To support the members in their professional practice. 

To practise as an engineer in Nova Scotia or to offer professional 
engineering services to the public in Nova Scotia, licensing by this body 
is a requirement. It also has implications for national and international 
mobility of engineering professionals. It is a member of the national 
body - Canadian Council of Professional Engineers (CCPE). CCPE in 
turn is a signatory to the Washington Accord that regulates the mutual 
recognition of professional qualifications in engineering among its 
signatories. 

Source: www.apens.ns.ca 



Governance and organizational structure 

Depending on the way the quality assurance agency is established 
and its administrative affiliation, its governance and organizational 
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Structure may vary. Commonly, the governance structure consists 
of an executive or governing body at the policy-making level that 
steers the policies and objectives of the quality assurance agency. 
The governing bodies may have members appointed, nominated or 
elected according to the rules of the organization. 

An important concern is the representation of the various higher 
education stakeholders in the governing body. The composition of 
the governing body is generally indicative of the relative power that 
the different stakeholders try to maintain over the agency. In agencies 
that are established by government, the government representative 
either sits in on or chairs the governing body. Some quality assurance 
agencies consider it important to have students, members of the 
public or end-users represented in the governing body. 

The composition also depends on other factors, such as the 
importance given to regional co-operation and the political will that 
supports such co-operation. International presence in the governing 
bodies of quality assurance agencies in Europe has been accelerated 
by the move towards the European Higher Education Area. In the 
Austrian Accreditation Committee (AAC), half of the members of 
the council are Austrians and the other half are experts from other 
European countries. International presence in review teams is 
more common than previously and the governing bodies of quality 
assurance agencies tend to have a favourable attitude towards it. This 
is due to the growing importance of regional dialogue among the 
quality assurance agencies and the internationalization of HEIs. In 
Europe, foreign experts are appointed to the accreditation committee/ 
council, as in the case of AAC (Austria), Akkreditierungsrat 
(Germany), AQAS (Ireland), AUQA (Australia), AAU (New 
Zealand) in Asia-Pacific, ACQUIN (Germany) and the Eoundation 
for International Business Administration Accreditation (EIBAA). 

The governing body may assume a variety of administrative, 
supervisory and decision-making functions. It also has the overall 
responsibility for the policies and functioning of the quality assurance 
agency. This means that the governing body may devolve authority to 
the head or director of the agency (appointed by the governing body 
itself), but will still be responsible for decision-making. The head of 
the agency can decide on the day-to-day administration, adhering 
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to the rules and regulations that govern the agency and reporting to 
the governing body. Any substantive changes or decisions would 
require the approval of the governing body. 

The governing body is in charge of the agency’s quality 
assurance decisions and monitors the credibility of the process. It 
also ensures that the quality assurance process results in thorough, 
informed and independent judgements. It may pay attention to the 
performance of the agency and guide its development. From time to 
time, members of the governing body may: (a) serve as members of 
a subcommittee or ad hoc working group; (b) attend events related 
to quality assurance matters on behalf of the agency; (c) speak to 
groups or in conferences about the work of the agency; (d) advise 
the staff of the agency based on the member’s specialist knowledge 
or experience; or (e) work with staff in areas of importance, such as 
papers and monographs on cross-border education. 



Box 17. Governance structure of a quality assurance 
agency (Ireland) 

Higher Education and Training Awards Council (HETAC), 
Ireland 

The appointment of the members of the council is set out in 
legislation. There are 15 members of the Council. The Chairman and the 
Chief Executive Officer are members of the Council. Other members 
are representative members appointed by the Minister for Education 
and Science (2), the Minister for Enterprise, Trade and Employment (1), 
and the Recognized Institutions (3). With the agreement of the Minister 
for Education and Science, one person is appointed as representative 
of learners and another person as representative of the employees of 
Recognized Institutions. Other members are nominated by the Irish 
Business and Employers Confederation (1) and the Irish Congress of 
Trade Unions (1). The Council once established may then nominate 
two additional members who have a special knowledge and experience 
related to the functions of the Council, one of which shall have 
international experience. The members of the Council operate on a 
part-time voluntary basis. The only full-time member of the Council is 
the Chief Executive Officer. The term of appointment is five years. 
Source: Vroeijenstijn, 2003. 
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Box 18. Governance structure of a quality assurance 
agency (Chile) 

National Commission for Accreditation, Chile 

In 1998, the Ministry of Education established a National 
Commission for Accreditation (CNAP), charged with designing an 
accreditation process and carrying it out. The commission has fourteen 
members, appointed by the Minister of Education, and a technical staff 
in charge of co-ordinating and managing accreditation procedures. 
This commission had as one of its main duties to design a permanent 
system for quality assurance, on the basis of their experience. 

The Commission prepared a proposal, which was then turned into 
a law project. Congress has already approved the proposed structure 
for the quality assurance agency, along the following lines; 

• The President of the country appoints the chair of the 
commission. 

• There are seven members, appointed by the main types of higher 
education institutions (public and private universities, professional 
institutes, technical training centres). 

• Two members are appointed by the national science and technology 
commission. 

• The Head of the HE Department of the Ministry of Education is 
also a member. 

• These members together decide on the appointment of two 
other persons, one representing employers, and one representing 
professional associations. 

• Two students are appointed by smdent organizations following 
specific guidelines set in the law. 

• The commission appoints a Secretary General, who heads the 
technical staff and has voice but no voting rights. 

All members of the commission act in a personal capacity, for a 
fixed period of four years, regardless of any changes in their institutional 
affiliation or current position. 

This governing body makes all policy decisions, and all 
accrediting decisions, which include institutional accreditation, the 
approval, registration and supervision of quality assurance agencies 
for programme accreditation and the accreditation of doctoral 
programmes. 

Source: Lemaitre, 2005. 
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Box 19. Responsibilities of the governing body (Australia) 

• The aim of the Board of AUQA is to implement the Objects of the 
company. 

• The Board sets the context for AUQA’s QA [quality assurance] 
activities, within which the activities are carried out by audit 
panels, guided by the staff. 

• In respect of audits, the Board is responsible for policies, 
procedures, people, and publishing. 

• The Board of AUQA acts with due diligence in relation to its task 
of corporate governance. 

Summary responsibilities of the Board 

• The Board determines policies consistent with the constitution. 

• The Board appoints auditors to the Register. 

• The Board Chair and the Executive Director set up audit panels 
from the Register, with the Chair acting on behalf of the Board. 

• In setting up an audit panel, the Board delegates to that panel the 
responsibility and authority for carrying out an audit according to 
the policies and procedures. 

• The Board approves the release of an audit report if it is substantial 
in content, convincing (in terms of the evidence presented), 
responsibly expressed, and consistent in tone and scope with 
AUQA’s responsibilities. 

Fuller expression of functions of the Board 

The functions of the Board are to: 

• Take responsibility for the performance of the organization, with 
respect to meeting the objects of the company; 

• Plan the strategic direction for AUQA, having in mind the national 
and international context, and within the Objects defined by the 
constitution; 

• Determine the policies of AUQA, within the parameters set by the 
constitution; 

• Monitor the implementation of the policies by the Executive 
Director and other staff; 

• Confirm that the operating procedures of AUQA are carried out; 

• Appoint auditors to the Register; 

• Approve the release of audit reports; 

• Approve the budget of AUQA; 

• Appoint the Executive Director of AUQA; 
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• Accept responsibility for the financial performance and reports of 
AUQA; 

• Submit to the Members an annual report of AUQA, including the 
audited accounts; 

• Advise the members on the constitution of AUQA. 

Source: http://www.auqa.edu.au/aboutauqa/policies/003/index.shtml 



With regard to the organizational structure, some quality 
assurance agencies divide responsibilities up internally, into areas 
such as general administration, training reviewers, orienting HEIs, 
external relations, conducting assessment visits, documentation, 
information systems and weh-related activities. The specializations 
may he related to major functions of the quality assurance process 
or he a mixture of quality assurance processes. These functions 
may he handled hy a staff member or a unit dedicated to these 
responsibilities. 

Other quality assurance agencies have units that perform 
certain functions or units that are responsible for certain regions 
or (types of) institutions. For example, all responsibilities related 
to institution X (e.g. National University of Mexico), a group of 
institutions of a certain region (e.g. all universities and community 
colleges of Illinois), or type of institution (e.g. all teacher education 
colleges of the country) might be allocated to a specific staff or 
unit. If the agency exercises more than one type of assessment 
process, division of responsibility may be based on the type of 
assessment (e.g. programme assessment is to be handled by X and 
institutional assessment is to be handled by Y). Frequently, agencies 
combine these approaches. They need to find out how best to divide 
responsibilities in their specific context, and which support structure 
will work best. 

Resources of quality assurance agencies 

Funding a quality assurance agency is both a political and an 
operational issue. When the quality assurance agency has been set 
up by a governmental initiative to serve governmental functions, 
significant funding is usually derived from the government itself. In a 
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quasi- governmental structure where the agency has acloserelationship 
with the government, hut is administered hy autonomous governing 
structures, the initial funding may come from the government. The 
cost of the actual quality assurance activity, however, is frequently 
home hy the HEIs concerned, although governments often meet 
the accreditation costs of puhlicly-funded HEIs, in some cases hy 
providing this funding directly to the agency. 

If the agency is owned hy the HEIs themselves, the funding for 
the accrediting body and process may he derived from the institution 
itself. The quality assurance bodies that are dissociated from 
governmental initiatives and HEIs, for instance those that function 
as NGOs and professional accreditation bodies, have to depend on 
the fees they charge for the assessment services. 

Considering these different cases, and the expenditures involved 
in running an agency, there are different funding schemes that 
involve either one or a combination of the following: 

• Governmental funding. In government-initiated systems, 
government provides at least the initial funding, and will 
normally cover at least part of the expenses related to the 
running of the agency. 

• Fees from HEIs. In many systems, institutions pay for the 
services received. This payment would normally cover all 
expenses related to the external review, plus the cost of training 
activities. 

• Income received by the agency for services rendered to 
institutions or organizations other than those applying for 
accreditation, such as conferences, workshops, consultancy, 
etc. 

The size of the agency, the budget, the volume of the activity 
and coverage differ greatly, varying from eight universities and two 
full-time staff to 16,000 HEIs and 20 full-time staff. Eor example 
in the Philippines, where each accrediting agency has a secretariat 
headed by an executive director, the secretariat of the Philippine 
Accrediting Association of Schools, Colleges and Universities 
(PAASCU) has the most staff. PAASCU serves the largest number 
of educational institutions, with 289 member institutions in 
1997 implementing 12 academic accreditation programmes. The 
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Accrediting Agency of Chartered Colleges and Universities in the 
Philippines (AACCUP) has only 48 memher institutions and the 
least number of academic accreditation programmes. Accordingly, 
it has fewer staff in the secretariat. Funding requirements vary from 
a few thousand US dollars to more than 1 million per annum. This 
depends not only on the size of the agency hut also on the salary 
structure and volume of activities, which themselves are dependent 
on the level of involvement of the agency in the quality assurance 
process. 

In addition to the number of staff, their competencies have to 
match the role they are expected to play in the quality assurance 
process. In some quality assurance agencies, members of staff take 
an active role in assessment, such as in report-writing, and they 
participate as experts in training programmes for reviewers and 
orientation courses for HEIs. Such agencies require competent and 
experienced staff who would be regarded by the academic community 
as peers. In other words, at least senior staff members of the agency 
need to have the profile of peers or reviewers as mentioned earlier. 
If the agency carries out programme accreditation, it might wish to 
choose members of staff who have those specializations. In the case 
of the agency staff who carry out administrative support functions 
only, their profile and competencies would vary accordingly. 



Box 20. Resources needed for the quality assurance agency 
(United States) 

Regional accrediting agencies typically have a limited number of staff, 
perhaps 12 to 20 in all, and an annual operating budget of $2 million 
to $3 million, mainly covering salaries, office expenses and the 
holding of two or three meetings each year. Professional staff have 
varied backgrounds but generally have doctoral-level education, have 
worked at academic institutions, have good organizational skills, and 
are circumspect and professional in demeanour. With programme 
accrediting, staff often possess degrees and training specific to the 
professional area of interest, for example, in engineering or nursing. 
Source: El-Khawas, 2001. 
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Along with institutional affiliation and funding, the quality 
assurance agency inherits certain expectations and terms of reference 
for its functioning from its providers or authorities. Consequently, 
the agency has to project its resource requirements - financial and 
human - depending on the range and quantity of activities it is 
expected to take on, and the role the agency staff are to play in those 
activities. 

Information system 

One of the main features of external quality assurance is the 
periodicity of the quality assurance process. Quality assurance 
agencies maintain records related to quality assurance outcomes 
- self-assessment reports that form the basis upon which to make 
decisions; recommendations of the review team; institutional 
responses, if any; decisions of the agency; appeals, if any; and their 
outcomes during this period. When making its final decisions, the 
agency must have at its disposal: information provided hy HEIs; 
information considered hy reviewers; evidence to justify decisions; 
and data concerning the process. All this information has to he well 
organized, especially when decisions can he challenged and if the 
outcome could have serious implications. Such handling of data is 
necessary to support the assessment process. 

There is another type of information system used hy some 
agencies - a public information system related to quality assurance 
outcomes. To ensure that the information provided by HEIs is reliable, 
that the information considered by the reviewers is appropriate, that 
the data used to make judgements are valid, and that meaningful use 
is made of the data collected, some agencies extend their services 
beyond supporting the assessment functions and make as much 
data as possible available to the public and other national databases. 
Critical data given in the institutional reports may feed into other 
national databases, and some institutional data may be derived 
from other national databases for further consideration or for cross 
validation of data provided by the HEIs. However, it should be noted 
that much of the information gathered through self-assessment by 
the HEIs will only be made available for assessment purposes. In 
many cases, however, HEIs will provide sensitive and strategic 
information only when they are sure that it will not be published, 
but used only for evaluation. 
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For this reason, most quality assurance agencies generally 
separate these two aspects - information handling to support 
assessment functions, and information handling for public 
disclosure. The provision of accurate, valid and reliable information 
to public information systems may be one of the requirements for 
accreditation, but this information will be different from what is 
required to make decisions about the quality of the institution (even 
if at least part of the two sets of information overlap). 

Credibility and accountability of the agency 

As the higher education institutions whose work they assess, 
quality assurance agencies have to be accountable for the quality 
of their work and they are obliged to demonstrate publicly that 
the quality assurance process implemented achieves the desired 
objectives effectively. To this end, they become accountable to 
many stakeholders to prove the credibility of the process and 
to ensure the objectivity and transparency of their decisions or 
recommendations . 

The credibility of the quality assurance process is a combination 
of many different factors and includes: clarity in policies; 
appropriateness of the quality assurance framework; transparency 
of the procedures; integrity of the people involved; and the desired 
impact on the system. 

To guarantee the credibility of the quality assurance process, 
the agency will have to ensure that its strategies include elements 
such as: (a) broad involvement of FIEIs in evolving the norms and 
criteria; (b) consensus-building to ensure widespread support; 
(c) careful development of the methods and instruments of 
assessment; (d) transparency in all policies and practices; (e) rigorous 
implementation of procedures; and (f) safeguards to enhance the 
professionalism of assessment. In systems that focus on quality 
control, the agency may have to choose a different set of elements to 
eliminate low-quality provisions. In this case, holding consultations 
with the higher education community on criteria will not be 
appropriate if the outcome of the quality assurance could be used to 
close down poor quality institutions. However, the agency can ensure 
credibility through clear and transparent procedures, rigorously 
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applied, with a good appeal system. Lots of training and working 
with the institutions can contrihute to the agency’s reputation in 
the academic community. In such a situation, the agency may not 
he able to ensure consensus among the HEIs, hut it can take into 
account any worthwhile comments to finetune its quality assurance 
procedures. 

Quality assurance agencies strive to develop a framework that 
is responsive to change hut consistent in its approach. In particular, 
the framework needs to he sensitive to the local context as well 
as on a par with international developments in its core elements. 
Once the framework is well in place, quality assurance agencies 
use various methods to select experts who are known for their 
integrity and competence. The qualifications and skills of the people 
who constitute the assessment teams are critical to the credihility 
of the whole process. Nonetheless, the professionalism with which 
the accreditation process is planned and implemented hy the 
accreditation agency is of equal importance to the success of the 
review team. Even the most highly qualified team can he thwarted 
in its work if the accreditation agency is not clear in its expectations 
of the team. 

Team composition is another area that needs attention. It is 
not possible for a single reviewer to be familiar with all the aspects 
of the functioning of an HEI or programme. Every individual’s 
perception of quality is influenced by factors that are beyond the 
control of the quality assurance agency. It is not possible for an 
agency to banish all the individual perceptions concerning a training 
programme, however rigorous it might be. An important concern 
is thus to ensure that the team will produce a good collective team 
assessment. A mixture of skills and experience lends greater fairness 
to the assessment. 

Reflection on the confiict-of-interestpolicy needs to be mentioned 
here. Although the reviewers are experts known for their integrity, 
in order to ensure objectivity most agencies require the reviewers to 
certify that they have no involvement with the proposed institution, 
directly or indirectly through any close relatives, in the past or at 
present, as either an employee or a member of any official body as 
a consultant or graduate. The conflicts of interest that the agency 
staff are likely to encounter should also be addressed appropriately. 
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These measures strengthen the credibility of the quality assurance 
agency. 

Involving HEIs in implementing the quality assurance procedures 
will also help establish much-needed credibility. In some countries, 
the institutions do not have any say in the constitution of the review 
team. Consulting the institutions when constituting the team is 
considered good practice to uphold the spirit of quality assurance 
as an exercise in partnership. Besides, it is of little use to send in 
a team whose judgement will not be accepted by the institution. It 
would only strengthen compliance without promoting improvement. 
Collecting feedback from the HEIs and those involved in the quality 
assurance process, and using it to further improve quality, also 
contributes to steering the agencies’ progress in the right direction. 
Having a sound appeal mechanism in place is another step that the 
quality assurance agencies frequently use to ensure their credibility 
and accountability. 

In other words, clarity in objectives, a framework for the 
evaluation, adherence to the framework, appropriate safeguards to 
ensure objectivity, consideration of the HEIs as equal partners and 
winning the confidence of the stakeholders can all greatly contribute 
to the credibility and acceptance of the quality assurance process. 

There are also mechanisms to ensure that the quality assurance 
agency explicitly demonstrates its accountability. A few ways in 
which agencies demonstrate their accountability are illustrated in 
the following pages. 

Accountability measures 

There is a strong relationship between ownership of the agency 
and the way accountability is ensured. It has been mentioned 
already that in the case of agencies established by governments, 
governmental officials (such as a representative from the ministry 
of education) tend to sit in on, or even chair the governing body. 
In addition, the government may have various mechanisms, such 
as calling for annual reports, to monitor the agency’s performance. 
When owned by the HEIs, the agency becomes accountable to the 
HEIs. There are wide variations in the ways the accountability 
measures are implemented, some of which are discussed below. 
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Built-in checks in the functioning of the quality assurance agency 
create an element of accountability 

There are many built-in checks that ensure that stakeholders 
are involved in shaping and monitoring the quality assurance 
processes. Representation of the various stakeholders, especially 
the cross-section of academia in the governing bodies, is one way 
of ensuring that the agency remains accountable. As mentioned in 
previous sections , some agencies have a representative from the public 
or user end in the governing board. Some quality assurance agencies 
have elaborate procedures to nominate the public representative to 
serve on the governing board. In general, quality assurance agencies 
submit their plans and annual reports to the governing bodies, which 
may even be made public to ensure transparency and strengthen 
accountability. Having an international presence in the governing 
bodies is another way of ensuring that the policies and procedures 
are compatible with practices in other countries and adds an element 
of comparability. 

BAN-PT, the accreditation agency of Indonesia, due to 
its affiliation to the government, submits annual reports to the 
government, which also allocates an annual budget that guides the 
activities of the agency to a certain extent. The Ontario Council on 
Graduate Studies (OCGS), established by the Council of Ontario 
Universities (COU), has the mandate to report in writing to COU at 
least once a year on its activities of the past year. 

An umbrella organization may look into the accountability 
of the quality assurance agencies 

Private quality assurance agencies, since they fulfil a public 
service function, frequently need to obtain recognition from an 
umbrella body as an accountability measure. This is the case in 
countries as diverse as the United States, the Philippines and 
Germany. In the United States, the regional accrediting bodies, as well 
as the professional accreditation agencies established by the HEIs, 
seek recognition either from the CHEA or from the United States 
Department of Education. Although seeking recognition by these 
bodies is voluntary, federal funds such as student support will flow 
only to institutions that are accredited by an agency that is recognized 
by CHEA. Accrediting bodies that seek recognition by the CHEA 
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must demonstrate that they meet the CHE A recognition standards. 
Accrediting organizations are expected to advance academic 
quality, demonstrate accountability, encourage improvement, 
employ appropriate procedures, continually reassess accreditation 
practices, and possess sufficient resources. The recognition review 
has an evaluation procedure similar to the accreditation exercise of 
the HEIs, with self-study and external review. During the process 
(which lasts six years), there are even sessions that are open to the 
public. 

AUQA, for instance, is responsible for conducting quality 
audits of state and territory government accreditation authorities on 
a five-year cycle. Action taken in response to audit reports are then 
the responsibility of the relevant department and minister. Audit 
by the AUQA serves as an accountability check for the state and 
territory government accreditation authorities. 

In Germany, an umbrella body, the Akkreditierungsrat, was 
created in 1998 to provide recognition to those private non-profit 
accreditation agencies that offer their services to German higher 
education institutions. These agencies are recognized, or not, on the 
basis of the criteria laid out in Box 21. 



Box 21. Basic standards for accrediting accreditation 
agencies by the German Akkreditierungsrat 

Accreditation agencies can be accredited by the Akkreditierungsrat if 
they meet principles and basic standards as follows: 

Accreditation agencies must be institutionally independent 
of higher education institutions as well as of business, industry and 
professional associations, and must perform accreditation procedures 
accordingly. The agencies must ensure that higher education 
institutions and representatives of professional practice are given 
appropriate opportunities to participate in the accreditation decision- 
making process. 

Accreditation agencies require an adequate staffing facility 
and funding infrastmcture reliably ascertained for a medium-term 
perspective. They operate on the basis of the principles of efficiency 
and economy, and will not be profit-oriented. 
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Accreditation agencies perform accreditation for all types of 
higher education institutions, since pursuant to § 1 9 German Framework 
Law on Higher Education and in accordance with KMK (Conference 
of Ministers of Education) and HRK (German Rectors’ Conference) 
resolutions, both universities and universities of applied sciences 
(Fachhochschulen) are entitled to establish degree programmes leading 
to BakkalaureusfBachelofs and Master’s degrees. 

Accreditation agencies must bring together national and 
international competence from all kinds of higher education 
institutions and should perform accreditation with respect to all types 
of programmes and disciplines. As an essential factor to evaluate the 
professional qualihcation of accreditation agencies, such approaches 
should be reflected, inter alia, in the recruitment of experts and in the 
design of evaluation procedures. The competence of accreditation 
agencies will also be validated by the documentation of evaluation 
criteria and standards, as well as staff qualifications. 

Accreditation agencies must prove by appropriate evidence that 
procedures followed in processes of programme accreditation are 
comprehensible and characterized by transparency. They must provide 
for internal measures of quality control and suitable practices of 
documentation and information (cf. the Akkreditie rungs rat Principles 
and Criteria for the Accreditation of Degree Programmes). 

Accreditation agencies are accountable to the Akkreditierungsrat 
also after they have been accredited. In particular, they are obliged to 
inform the Akkreditierungsrat without delay of any degree programme 
for which accreditation status has been extended by them, and to 
submit an annual activity report. 

Source: www.accreditation-council.de/ 



Voluntary co-ordination through regional networks and adherence 
to their standards and criteria 

It has become common practice that quality assurance agencies 
voluntarily join together to form networks and follow commonly 
agreed principles or practices. Although accountability may not 
be their motivation for joining networks, adherence to common 
standards and criteria simultaneously enhances their credibility and 
serves to demonstrate the agency’s accountability. Good practices 
developed by the INQAAHE and the European Network for Quality 
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Assurance (ENQA) are examples of this. Box 22 presents some 
of the good practices listed hy the INQAAHE, with reference to 
accountability to the public. 



Box 22. Guidelines for good practices: INQAAHE 

In its work, the EQA Agency informs and responds to the public in 
accordance with the legislation or cultural context relating to the 
agency. This includes making public and explicit its documentation 
e.g. policies, procedures and criteria. 

The agency also demonstrates public accountability by reporting 
openly on its review decisions and making the outcomes of the 
evaluation public in a way appropriate to the relevant country legislation 
and the type of review undertaken. The content of the public report 
may differ depending on the cultural context and will also depend on 
the requirements set for accountability. 

Source'. INQAAHE (www.inqaahe.org). 



Periodic assessment of agencies is a way to demonstrate 
accountability 

There is a growing awareness of the benefits of meta-evaluation 
or ‘evaluating the evaluation itself’ as a critical measure to ensure 
accountability, continuous improvement and development of good 
practices of quality assurance agencies. European quality assurance 
agencies are expected to submit themselves to a cyclic review 
every five years. Some agencies conduct impact studies and mid- 
term reviews that contribute to understanding the progress towards 
realization of the objectives. Some agencies have invited international 
experts to observe assessment visits, held training programmes, and 
organized consultations to provide feedback. The Higher Education 
Quality Committee (HEQC) of South Africa has an International 
Reference Group that consists of three international members that 
act as a sounding board for its development. The AUQA and national 
commission for evaluation in Chile are in the process of undergoing 
external reviews. 
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Registering of agencies to ensure accountability 

Although this measure is yet to become functional, there 
is ongoing discussion about how to do so. The quality assurance 
agencies of the European region have agreed to register external 
quality assurance agencies operating in Europe. Box 23 illustrates 
how this register is likely to take shape. 



Box 23. Standards and guidelines for quality assurance 
in the European higher education area: ENQA 

ENQA committed itself before the Berlin Ministerial meeting of 2003 
to develop, in co-operation with the relevant stakeholders, a European 
register of quality assurance agencies, covering public, private, and 
thematic agencies, operating or planning to operate in Europe. 

The register would meet the interest of higher education institutions 
and governments in being able to identify professional and credible 
quality assurance agencies operating in Europe. The interest has firstly 
its basis in the complicated area of recognition of non-national degrees. 
Recognition procedures would be strengthened if it were transparent to 
what extent providers were themselves quality assured by recognized 
agencies. Secondly, it is increasingly possible for higher education 
institutions to seek quality assurance from agencies across national 
borders. Higher education institutions would of course be helped in 
this process by being able to identify professional agencies from a 
reliable register ... 

... A European Register Committee will decide on admissions 
to the European register. The committee will use agency compliance 
with the European standards for external quality assurance agencies as 
identified in the cyclical review as one criterion for placement in the 
register. Other criteria should be developed, which will take account of 
the diversity of the higher education systems. 

Source: Standard and guidelines for quality assurance in the European Higher Education 
Area (see: www.enqa.net). 



The International Association of University Presidents (lAUP)’s 
and INQAAHE’s proposal in 1999 to create a quality label similar 
to the quality register in order to identify credible external quality 
assurance agencies met with opposition. INQAAHE consequently 
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dropped the idea of a quality label, but instead developed a set of 
good practices, which were discussed earlier. 

The above discussions show very clearly that to carry out 
quality assurance, its implementation needs to be adapted to the 
local context, at the same time as it also requires international/ 
regional compatibility. All quality assurance has to respond to both 
imperatives, and tries to do so through the available options, many 
of which have been discussed in this booklet. 
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New contextual factors arising from both globalization and the 
internal transformation of higher education systems make quality 
assurance an ever more important function for public authorities. 
The development of quality assurance systems appears to be one of 
the major trends in higher education policy and finds itself reinforced 
through the creation of regional networks of quality assurance 
agencies. Such networks help agencies to exchange experiences and 
develop codes of practice. 

Within this overall similarity between EQA systems, the 
above discussions have shown the various options in setting them 
up. Options cannot be discussed without taking into account the 
particular national policy context of a higher education system, its 
traditions and culture. 

First, we can conclude that EQA systems tend to fill gaps in the 
broader quality assurance system, and that they focus on functions 
that are not yet carried out by other agencies. This explains many 
of the differences found in a comparative analysis of EQA systems. 
Another source of divergence lies in the specific academic traditions 
and culture of a country. Specific procedures have to be seen as 
legitimate within a given system, and what is legitimate varies from 
one context to the other. Variations in the understanding of what 
constitutes institutional autonomy, for instance, lead to differing 
understandings of how an effective quality assurance system should 
operate. 

Second, other structural factors equally condition the option 
chosen. The size of the higher education system is one of the 
contextual issues that may impact heavily on the choices that are made 
regarding structure. Massive systems with numerous institutions 
will naturally develop a different mechanism for quality assurance 
than small developing states. Obviously, running a quality assurance 
system in India is not the same as running one in the Seychelles. 
The size of the system to be covered by the process might vary from 
a few thousand programmes to only a few institutions. However, 
the size of the system does not necessarily influence the choice of 
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unit for quality assurance since most systems have both institutional 
and programme level quality assurance arrangements. In this case, 
agencies tend to specialize in one mechanism: While one agency may 
concentrate on institutional quality assurance, another may look at 
the programmes. Some agencies do both. However, what is certain 
is that the size of the education system to be covered influences other 
aspects of quality assurance: Policies and practices related to the 
participation of agency staff in site visits, the selection of reviewers 
and constitution of the review team, and the place given to the training 
of reviewers are shaped partly by the size of the system. For example, 
in large systems there is a heavy reliance on external reviewers with 
limited support from the quality assurance body. Consequently, the 
external visit requires external reviewers who are competent enough 
to adhere to the quality assurance framework but with minimum 
direct guidance from the staff. Identifying external reviewers may 
not be a big challenge in large systems, but training them to adhere 
to the quality assurance framework in a consistent manner, putting 
in place appropriate safeguards to minimize inter-team variances, 
and ensuring professionalism in such large operations may become 
very challenging. 

The level of private provision, the stage of development 
they have reached in the country, and the public perception of 
that provision all influence the quality assurance arrangement 
in a country. For example, whether the same agency covers both 
private and public provisions and whether the same standards are 
applied to both depends on the way quality assurance is developed 
in the national context. As mentioned already, quality assurance 
systems are established to fill the gaps in the existing framework 
of the country. As private provision grows in a country, national 
quality assurance arrangements are strengthened, depending on 
the challenges that emerge, sometimes by expanding the scope 
of the existing arrangements, and sometimes by establishing new 
structures. For example, there are economies where the emergence 
of private sector institutions has resulted in the need for an explicit 
quality assurance mechanism to address specifically private 
provisions. While the ministries had control (often funding-related) 
over the publicly-funded institutions and programmes, the need for a 
parallel mechanism to assure quality in the private sector that did not 
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demand or have access to public funds resulted in the establishment 
of quality assurance structures that focused only on the private sector. 
Malaysia, Singapore and Hong Kong are typical examples. More 
recently in Malaysia, it was decided that quality assurance should 
cover both public and private provisions of higher education. 

The maturity of the system is another factor that conditions the 
role that the various stakeholders can play in the quality assurance 
arrangement. In mature systems, HEIs play a major role by 
contributing to the development or strengthening of the framework. 
They shape the quality assurance developments in the country. Getting 
competent reviewers from the national higher education sector may 
not be a problem. However, in developing systems, where there may 
only be a small core group of competent people to begin with, a lot 
of steering and decision-making has to be shouldered by the agency 
itself, until capacity is developed in the system. Accordingly, there 
may be reliance on international expertise to take on responsibility 
for quality assurance, or the staff may have a substantial role in the 
process. In mature systems, with appropriate training, the reviewers 
(who come from the higher education system) will be able to act on 
behalf of the agency and fulfil their responsibility with minimum 
direction from the agency. 

A third major conclusion of this booklet is that the overall 
purpose of an EQA system corresponds to a specific approach. This 
approach conditions whether the system should be compulsory or 
voluntary, whether standards or the ‘fitness for purpose’ concept is 
used, and whether accreditation or audit mechanisms predominate. 
However, it is also quite common for EQA agencies to emerge 
as multi-functional systems, and embrace new functions such as 
licensing, institutional audit and programme accreditation, often 
including recognition of foreign credentials. 

The structure of EQA usually corresponds to the overall 
philosophy of the system (accountability or conformity versus 
quality improvement and development of the system). When 
quality improvement is the aim, then a voluntary mechanism is a 
better option than a compulsory one. The assumption is that only 
when higher education institutions are motivated and committed 
to change can the EQA system operate as a development tool for 
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higher education. Strong academic commitment is needed for EQA 
to become an instrument for quality enhancement in all cases. 
However, it may also he necessary to put in place a system of quality 
assurance oriented towards controlling minimum standards when it 
is known that there are many low-quality providers in the system. 

In accordance with the basic underlying purpose of EQA systems, 
it needs to be decided whether quality assurance will be organized as 
quality assessment, quality audit or accreditation. Quality assessment 
is the more improvement-oriented, developmental approach through 
which the strengths and weaknesses of a higher education institution 
or programme are assessed. Quality audit, which focuses on the 
system for internal quality control, is more development-oriented 
and thus more appropriate when institutions and their programmes 
are of relatively even quality and have matured. Accreditation, which 
in fact imposes a cut-off point as to what is acceptable and what is 
not, is more appropriate for quality control purposes. The need to 
practise quality control in an emerging and rapidly growing private 
sector of higher education and ensure comparability of credentials 
has created pressure to develop standards-based approaches to EQA, 
and accreditation has thus become the favoured approach to EQA. 

This booklet has also discussed the role and functioning of the 
support structure for quality assurance: the agency. The issue of 
autonomy, from both state bureaucracy and the academic community, 
has been outlined as one of the most crucial concerns when setting 
up a quality assurance system. In addition, the need to develop trust 
in the quality assurance mechanism has been pointed out throughout 
the booklet. Trust can be gained through transparency and enhanced 
through many different devices, such as active dissemination of 
information to the public, both of quality assurance instruments 
and reports. The agency plays a crucial role in developing trust in 
the system through the way it puts in place and runs the quality 
assurance process. 

Quality assurance has become a somewhat fashionable item on 
national policy agendas for higher education. This booklet should 
conclude by emphasizing three points of caution: 

• Eirst, quality assurance is not an aim in itself. It is an instrument 

through which the state, directly or through delegation, may both 
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enact its role to protect students and families from low-quality 
or fraudulent providers and serve the purpose of quality 
improvement in academic departments and institutions. Quality 
assurance should thus he conceived as a support mechanism for 
the enhancement of higher education. 

Second, quality assurance has a cost: hoth financial and human. 
In addition to the financial resources necessary to implement the 
different process elements it entails, it requires much academic 
staff time, which could possibly also he used for core academic 
activities, such as teaching and research. Thus, quality assurance 
will only survive in the end if it proves to he an effective tool to 
regulate higher education and improve its functioning. 

Third, the existence of a quality assurance mechanism does not 
automatically mean that national higher education provision is 
of good quality. If higher education systems are grossly lacking 
in human resources, infrastructure and equipment, as is the case 
in many poor countries, quality assurance as such is not the 
solution. It can help point out existing problems and resource 
needs, but it is not the answer to the shortage of resources 
experienced in many countries. 
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